- From: Michael S Elledge <elledge@msu.edu>
- Date: Mon, 25 Jun 2012 10:34:13 -0400
- CC: Alistair Garrison <alistair.j.garrison@gmail.com>, RichardWarren <richard.warren@userite.com>, Eval TF <public-wai-evaltf@w3.org>
Hi All-- I agree with Alistair. We nearly always test a sample of pages in a website. Although it would be ideal to test every page in a site, it is impractical because of time and cost, especially if it is performed manually. Many people reading our methodology will be looking to apply it to their reviews, which out of necessity will be based on sampling. The alternative, relying solely on automated checkers to review a medium to large site in its entirety, I think we can all agree is not a viable alternative, even with their improvements. We spent a significant amount of time describing sampling approaches early in this process, so I'm surprised that the "all or nothing" approach is still being debated. I may have missed something along the way, however, so please forgive me if I did. Best regards, Mike On 6/25/2012 3:08 AM, Alistair Garrison wrote: > Hi Richard, > > Reading the archive I see we have talked around the subject of sampling - but not actually whether to evaluate all pages instead of a sample. Reading a number of emails, however, it becomes clear that we all seem to use some kind of sampling effort - hence its seemingly automatic acceptance to this point. > > To my mind, there are many reasons for adopting our reasonably straight-forward sample-based approach (again we have all mostly done something similar for years), even for smaller sites, over evaluating all pages. I suppose its lower cost in terms of time / effort - with the same actual benefits is one of the top reasons for sampling. > > I'm also worried that the changes you suggest (did it also need a change to the Requirements docs) at this stage will create a two-tier (all or sample) approach, forking our current work and possibly opening a big can of worms (like how do you realistically, and with very high confidence, find all pages in a website, what exactly is a small or medium site, etc...). > > I remain to be convinced, but I would be interested to hear the views of others. > > All the best > > Alistair > > On 22 Jun 2012, at 12:05, RichardWarren wrote: > >> Reason for making the default position to include all pages (entire website) >> >> 1) Taking the Internet (WWW) as a whole, the majority of sites are quite small (100 or so pages), typically things like "Mum& Pop" stores, SME profiles, personal or project websites. >> >> 2) Where this is practical a full evaluation is more reliable than a sample. >> >> 3) Our brief is to deliver an evaluation methodology, not a sampling methodology. >> >> 4) Reliable sampling is a complex procedure, if owners of small/medium sites think they have to go through sampling they will give up. >> >> 5) Sampling procedure will only be required for large sites so it should be an option. The default should be to evaluate the whole site. If the evaluator feels that is too large a task then s/he should have the option to use a sampling procedure to help manage the evaluation work load. >> >> My feeling as that we need to change the order of our text so that sampling is offered as the option, not the full audit. >> >> Richard >> >> -----Original Message----- From: Alistair Garrison >> Sent: Friday, June 22, 2012 10:39 AM >> To: RichardWarren ; Eval TF >> Subject: All pages >> >> Hi Richard, >> >> We were not able to debate the agenda item relating to "testing all pages"? Can you just remind me what was behind this issue? >> >> All the best >> >> Alistair >> > > -- Michael S. Elledge Associate Director Usability/Accessibility Research and Consulting Michigan State University Kellogg Center 219 S. Harrison Rd Room 93 East Lansing, MI 48824 517-353-8977
Received on Monday, 25 June 2012 14:34:47 UTC