Re: All pages

I agree with this approach too. The default (and ideal) would be to 
check all pages. In cases where this is not practically feasible we 
provide a robust sampling procedure.

This probably affects several sections, including the introduction, 
though rather editorially only.

Regards,
   Shadi


On 25.6.2012 19:47, RichardWarren wrote:
> Michael,
> We are not suggesting "all or nothing" .
> We are saying that the preferred method is to validate all pages, but if
> this is too large a task (which for typically large sites it will be) then
> here is a sampling procedure which will ensure that all important elements
> are covered.
>
> Thus owners of small sites that want to check their compliance can skip the
> sampling process and get straight on with the method of validating their
> site.
>
> Richard
>
> -----Original Message----- From: Michael S Elledge
> Sent: Monday, June 25, 2012 3:34 PM
> To: public-wai-evaltf@w3.org
> Cc: Alistair Garrison ; RichardWarren ; Eval TF
> Subject: Re: All pages
>
> Hi All--
>
> I agree with Alistair. We nearly always test a sample of pages in a
> website. Although it would be ideal to test every page in a site, it is
> impractical because of time and cost, especially if it is performed
> manually. Many people reading our methodology will be looking to apply
> it to their reviews, which out of necessity will be based on sampling.
> The alternative, relying solely on automated checkers to review a medium
> to large site in its entirety, I think we can all agree is not a viable
> alternative, even with their improvements.
>
> We spent a significant amount of time describing sampling approaches
> early in this process, so I'm surprised that the "all or nothing"
> approach is still being debated. I may have missed something along the
> way, however, so please forgive me if I did.
>
> Best regards,
>
> Mike
>
> On 6/25/2012 3:08 AM, Alistair Garrison wrote:
>> Hi Richard,
>>
>> Reading the archive I see we have talked around the subject of
>> sampling - but not actually whether to evaluate all pages instead of a
>> sample. Reading a number of emails, however, it becomes clear that we
>> all seem to use some kind of sampling effort - hence its seemingly
>> automatic acceptance to this point.
>>
>> To my mind, there are many reasons for adopting our reasonably
>> straight-forward sample-based approach (again we have all mostly done
>> something similar for years), even for smaller sites, over evaluating
>> all pages.  I suppose its lower cost in terms of time / effort - with
>> the same actual benefits is one of the top reasons for sampling.
>>
>> I'm also worried that the changes you suggest (did it also need a
>> change to the Requirements docs) at this stage will create a two-tier
>> (all or sample) approach, forking our current work and possibly
>> opening a big can of worms (like how do you realistically, and with
>> very high confidence, find all pages in a website, what exactly is a
>> small or medium site, etc...).
>>
>> I remain to be convinced, but I would be interested to hear the views
>> of others.
>>
>> All the best
>>
>> Alistair
>>
>> On 22 Jun 2012, at 12:05, RichardWarren wrote:
>>
>>> Reason for making the default position to include all pages (entire
>>> website)
>>>
>>> 1) Taking the Internet (WWW) as a whole, the majority of sites are
>>> quite small (100 or so pages), typically things like "Mum&  Pop"
>>> stores, SME profiles, personal or project websites.
>>>
>>> 2) Where this is practical a full evaluation is more reliable than a
>>> sample.
>>>
>>> 3) Our brief is to deliver an evaluation methodology, not a sampling
>>> methodology.
>>>
>>> 4) Reliable sampling is a complex procedure, if owners of
>>> small/medium sites think they have to go through sampling they will
>>> give up.
>>>
>>> 5) Sampling procedure will only be required for large sites so it
>>> should be an option. The default should be to evaluate the whole
>>> site. If the evaluator feels that is too large a task then s/he
>>> should have the option to use a sampling procedure to help manage the
>>> evaluation work load.
>>>
>>> My feeling as that we need to change the order of our text so that
>>> sampling is offered as the option, not the full audit.
>>>
>>> Richard
>>>
>>> -----Original Message----- From: Alistair Garrison
>>> Sent: Friday, June 22, 2012 10:39 AM
>>> To: RichardWarren ; Eval TF
>>> Subject: All pages
>>>
>>> Hi Richard,
>>>
>>> We were not able to debate the agenda item relating to "testing all
>>> pages"? Can you just remind me what was behind this issue?
>>>
>>> All the best
>>>
>>> Alistair
>>>
>>
>>
>

-- 
Shadi Abou-Zahra - http://www.w3.org/People/shadi/
Activity Lead, W3C/WAI International Program Office
Evaluation and Repair Tools Working Group (ERT WG)
Research and Development Working Group (RDWG)

Received on Tuesday, 26 June 2012 09:12:44 UTC