W3C home > Mailing lists > Public > public-wai-evaltf@w3.org > June 2012

Re: All pages

From: RichardWarren <richard.warren@userite.com>
Date: Mon, 25 Jun 2012 18:47:55 +0100
Message-ID: <4BFBA6F79D3540559FDF06C75C25B904@DaddyPC>
To: <public-wai-evaltf@w3.org>
We are not suggesting "all or nothing" .
We are saying that the preferred method is to validate all pages, but if
this is too large a task (which for typically large sites it will be) then
here is a sampling procedure which will ensure that all important elements
are covered.

Thus owners of small sites that want to check their compliance can skip the
sampling process and get straight on with the method of validating their


-----Original Message----- 
From: Michael S Elledge
Sent: Monday, June 25, 2012 3:34 PM
To: public-wai-evaltf@w3.org
Cc: Alistair Garrison ; RichardWarren ; Eval TF
Subject: Re: All pages

Hi All--

I agree with Alistair. We nearly always test a sample of pages in a
website. Although it would be ideal to test every page in a site, it is
impractical because of time and cost, especially if it is performed
manually. Many people reading our methodology will be looking to apply
it to their reviews, which out of necessity will be based on sampling.
The alternative, relying solely on automated checkers to review a medium
to large site in its entirety, I think we can all agree is not a viable
alternative, even with their improvements.

We spent a significant amount of time describing sampling approaches
early in this process, so I'm surprised that the "all or nothing"
approach is still being debated. I may have missed something along the
way, however, so please forgive me if I did.

Best regards,


On 6/25/2012 3:08 AM, Alistair Garrison wrote:
> Hi Richard,
> Reading the archive I see we have talked around the subject of sampling - 
> but not actually whether to evaluate all pages instead of a sample. 
> Reading a number of emails, however, it becomes clear that we all seem to 
> use some kind of sampling effort - hence its seemingly automatic 
> acceptance to this point.
> To my mind, there are many reasons for adopting our reasonably 
> straight-forward sample-based approach (again we have all mostly done 
> something similar for years), even for smaller sites, over evaluating all 
> pages.  I suppose its lower cost in terms of time / effort - with the same 
> actual benefits is one of the top reasons for sampling.
> I'm also worried that the changes you suggest (did it also need a change 
> to the Requirements docs) at this stage will create a two-tier (all or 
> sample) approach, forking our current work and possibly opening a big can 
> of worms (like how do you realistically, and with very high confidence, 
> find all pages in a website, what exactly is a small or medium site, 
> etc...).
> I remain to be convinced, but I would be interested to hear the views of 
> others.
> All the best
> Alistair
> On 22 Jun 2012, at 12:05, RichardWarren wrote:
>> Reason for making the default position to include all pages (entire 
>> website)
>> 1) Taking the Internet (WWW) as a whole, the majority of sites are quite 
>> small (100 or so pages), typically things like "Mum&  Pop" stores, SME 
>> profiles, personal or project websites.
>> 2) Where this is practical a full evaluation is more reliable than a 
>> sample.
>> 3) Our brief is to deliver an evaluation methodology, not a sampling 
>> methodology.
>> 4) Reliable sampling is a complex procedure, if owners of small/medium 
>> sites think they have to go through sampling they will give up.
>> 5) Sampling procedure will only be required for large sites so it should 
>> be an option. The default should be to evaluate the whole site. If the 
>> evaluator feels that is too large a task then s/he should have the option 
>> to use a sampling procedure to help manage the evaluation work load.
>> My feeling as that we need to change the order of our text so that 
>> sampling is offered as the option, not the full audit.
>> Richard
>> -----Original Message----- From: Alistair Garrison
>> Sent: Friday, June 22, 2012 10:39 AM
>> To: RichardWarren ; Eval TF
>> Subject: All pages
>> Hi Richard,
>> We were not able to debate the agenda item relating to "testing all 
>> pages"? Can you just remind me what was behind this issue?
>> All the best
>> Alistair

Michael S. Elledge
Associate Director
Usability/Accessibility Research and Consulting
Michigan State University
Kellogg Center
219 S. Harrison Rd Room 93
East Lansing, MI  48824
Received on Monday, 25 June 2012 17:48:25 UTC

This archive was generated by hypermail 2.3.1 : Tuesday, 6 January 2015 20:40:21 UTC