Re: All pages

Hi Shadi, Richard, 

Re-reading your emails you seem to be looking at conformance evaluation from a new viewpoint (maybe rightly / maybe wrongly?) from the one we seem to have adopted to date.

Viewpoint 1 (to date) is an evaluator wishing to see if a whole website conforms with WCAG 2.0.  

With regard to this viewpoint an evaluator doesn't need to check all pages, simply enough to confirm whether the website conforms or doesn't.  If there are several examples of content which pass a checkpoint and the same team has built all the site, I suppose you can assume that the other relevant instances of content will also be ok, and visa versa for content with issues.  You would not think about checking all items on a production line, would you?  
Historically most people seem to think this type of evaluation is efficiently and effectively achieved by sampling, which is reflected in their different methodologies.  Checking all pages could be seen as unnecessary 'overkill' even on small sites...   
 
Viewpoint 2 (new) is a website owner wanting to find and correct all faults in their website - in which case they would want to look at all pages no matter what the size.

Either way, treating the concept as editorial only sounds to me a bit casual, but I'd be interested to hear other views...

All the best 

Alistair

On 26 Jun 2012, at 11:12, Shadi Abou-Zahra wrote:

> I agree with this approach too. The default (and ideal) would be to check all pages. In cases where this is not practically feasible we provide a robust sampling procedure.
> 
> This probably affects several sections, including the introduction, though rather editorially only.
> 
> Regards,
>  Shadi
> 
> 
> On 25.6.2012 19:47, RichardWarren wrote:
>> Michael,
>> We are not suggesting "all or nothing" .
>> We are saying that the preferred method is to validate all pages, but if
>> this is too large a task (which for typically large sites it will be) then
>> here is a sampling procedure which will ensure that all important elements
>> are covered.
>> 
>> Thus owners of small sites that want to check their compliance can skip the
>> sampling process and get straight on with the method of validating their
>> site.
>> 
>> Richard
>> 
>> -----Original Message----- From: Michael S Elledge
>> Sent: Monday, June 25, 2012 3:34 PM
>> To: public-wai-evaltf@w3.org
>> Cc: Alistair Garrison ; RichardWarren ; Eval TF
>> Subject: Re: All pages
>> 
>> Hi All--
>> 
>> I agree with Alistair. We nearly always test a sample of pages in a
>> website. Although it would be ideal to test every page in a site, it is
>> impractical because of time and cost, especially if it is performed
>> manually. Many people reading our methodology will be looking to apply
>> it to their reviews, which out of necessity will be based on sampling.
>> The alternative, relying solely on automated checkers to review a medium
>> to large site in its entirety, I think we can all agree is not a viable
>> alternative, even with their improvements.
>> 
>> We spent a significant amount of time describing sampling approaches
>> early in this process, so I'm surprised that the "all or nothing"
>> approach is still being debated. I may have missed something along the
>> way, however, so please forgive me if I did.
>> 
>> Best regards,
>> 
>> Mike
>> 
>> On 6/25/2012 3:08 AM, Alistair Garrison wrote:
>>> Hi Richard,
>>> 
>>> Reading the archive I see we have talked around the subject of
>>> sampling - but not actually whether to evaluate all pages instead of a
>>> sample. Reading a number of emails, however, it becomes clear that we
>>> all seem to use some kind of sampling effort - hence its seemingly
>>> automatic acceptance to this point.
>>> 
>>> To my mind, there are many reasons for adopting our reasonably
>>> straight-forward sample-based approach (again we have all mostly done
>>> something similar for years), even for smaller sites, over evaluating
>>> all pages.  I suppose its lower cost in terms of time / effort - with
>>> the same actual benefits is one of the top reasons for sampling.
>>> 
>>> I'm also worried that the changes you suggest (did it also need a
>>> change to the Requirements docs) at this stage will create a two-tier
>>> (all or sample) approach, forking our current work and possibly
>>> opening a big can of worms (like how do you realistically, and with
>>> very high confidence, find all pages in a website, what exactly is a
>>> small or medium site, etc...).
>>> 
>>> I remain to be convinced, but I would be interested to hear the views
>>> of others.
>>> 
>>> All the best
>>> 
>>> Alistair
>>> 
>>> On 22 Jun 2012, at 12:05, RichardWarren wrote:
>>> 
>>>> Reason for making the default position to include all pages (entire
>>>> website)
>>>> 
>>>> 1) Taking the Internet (WWW) as a whole, the majority of sites are
>>>> quite small (100 or so pages), typically things like "Mum&  Pop"
>>>> stores, SME profiles, personal or project websites.
>>>> 
>>>> 2) Where this is practical a full evaluation is more reliable than a
>>>> sample.
>>>> 
>>>> 3) Our brief is to deliver an evaluation methodology, not a sampling
>>>> methodology.
>>>> 
>>>> 4) Reliable sampling is a complex procedure, if owners of
>>>> small/medium sites think they have to go through sampling they will
>>>> give up.
>>>> 
>>>> 5) Sampling procedure will only be required for large sites so it
>>>> should be an option. The default should be to evaluate the whole
>>>> site. If the evaluator feels that is too large a task then s/he
>>>> should have the option to use a sampling procedure to help manage the
>>>> evaluation work load.
>>>> 
>>>> My feeling as that we need to change the order of our text so that
>>>> sampling is offered as the option, not the full audit.
>>>> 
>>>> Richard
>>>> 
>>>> -----Original Message----- From: Alistair Garrison
>>>> Sent: Friday, June 22, 2012 10:39 AM
>>>> To: RichardWarren ; Eval TF
>>>> Subject: All pages
>>>> 
>>>> Hi Richard,
>>>> 
>>>> We were not able to debate the agenda item relating to "testing all
>>>> pages"? Can you just remind me what was behind this issue?
>>>> 
>>>> All the best
>>>> 
>>>> Alistair
>>>> 
>>> 
>>> 
>> 
> 
> -- 
> Shadi Abou-Zahra - http://www.w3.org/People/shadi/
> Activity Lead, W3C/WAI International Program Office
> Evaluation and Repair Tools Working Group (ERT WG)
> Research and Development Working Group (RDWG)
> 
> 
>

Received on Tuesday, 26 June 2012 09:56:55 UTC