Re: EvalTF discussion 5.5

Hi, 

The issue of "margin of error" relates to the size of the website and the number of pages actually being assessed.  I'm not so keen on the "5% incidental error" idea.

If you assess 1 page from a 1 page website there should be no margin of error.  
If you assess 10 pages from a 10 page website there should be no margin of error.
If you assess 10 pages from a 100 page website you will have certainty for 10 pages and uncertainty for 90.

Instead of exploring the statistical complexities involved in trying to accurately define how uncertain we are (which could take a great deal of precious time) - could we not just introduce a simple disclaimer e.g.

"The evaluator has tried their hardest to minimise the margin for error by actively looking for all content relevant to each technique being assessed which might have caused a fail."  

Food for thought...

Alistair

On 12 Jan 2012, at 10:04, Shadi Abou-Zahra wrote:

> Hi Martijn, All,
> 
> Good points but it sounds like we are speaking more of impact of errors rather than of the incidental vs systemic aspects of them. Intuitively one could say that an error that causes a barrier to completing a task on the web page needs to be weighted more significantly than an error that does not have the same impact, but it will be difficult to define what a "task" is. Maybe listing specific situations as you did is the way to go but I think we should not mix the two aspects together.
> 
> Best,
>  Shadi
> 
> 
> On 12.1.2012 09:41, Martijn Houtepen wrote:
>> Hi Eric, TF
>> 
>> I would like to make a small expansion to your list, as follows:
>> 
>> Errors can be incidental unless:
>> 
>> a) it is a navigation element
>> b) the alt-attribute is necessary for the understanding of the information / interaction / essential to a key scenario or complete path
>> c) other impact related thoughts?
>> d) there is an alternative
>> 
>> So an unlabeled (but required) field in a form (part of some key scenario) will be a systemic error.
>> 
>> Martijn
>> 
>> -----Oorspronkelijk bericht-----
>> Van: Velleman, Eric [mailto:evelleman@bartimeus.nl]
>> Verzonden: woensdag 11 januari 2012 15:01
>> Aan: Boland Jr, Frederick E.
>> CC: Eval TF
>> Onderwerp: RE: EvalTF discussion 5.5
>> 
>> Hi Frederick,
>> 
>> Yes agree, but I think we can have both discussions at the same time. So:
>> 1. How do we define an error margin to cover non-structuraal errors?
>> 2. How can an evaluator determine the impact of an error?
>> 
>> I could imagine we make a distinction between structural and incidental errors. The 1 failed alt-attribute out of 100 correct ones would be incidental... unless (and there comes the impact):
>>   a) it is a navigation element
>>   b) the alt-attribute is necessary for the understanding of the information / interaction
>>   c) other impact related thoughts?
>>   d) there is an alternative
>> 
>> We could set the acceptance rate for incidental errors. Example: the site would be totally conformant, but with statement that for alt-attributes, there are 5% incidental fails.
>> This also directly relates to conformance in WCAG2.0 specifically section 5 Non-interference.
>> 
>> Eric
>> 
>> 
>> 
>> ________________________________________
>> Van: Boland Jr, Frederick E. [frederick.boland@nist.gov]
>> Verzonden: woensdag 11 januari 2012 14:32
>> Aan: Velleman, Eric
>> CC: Eval TF
>> Onderwerp: RE: EvalTF discussion 5.5
>> 
>> As a preamble to this discussion, I think we need to define more precisely ("measure"?) what an "impact" would be (for example, impact to whom/what and what specifically are the consequences of said impact)?
>> 
>> Thanks Tim
>> 
>> -----Original Message-----
>> From: Velleman, Eric [mailto:evelleman@bartimeus.nl]
>> Sent: Wednesday, January 11, 2012 4:15 AM
>> To: public-wai-evaltf@w3.org
>> Subject: EvalTF discussion 5.5
>> 
>> Dear all,
>> 
>> I would very much like to discuss section 5.5 about Error Margin.
>> 
>> If one out of 1 million images on a website fails the alt-attribute this could mean that the complete websites scores a fail even if the "impact" would be very low. How do we define an error margin to cover these non-structural errors that have a low impact. This is already partly covered inside WCAG 2.0. But input and discussion would be great.
>> 
>> Please share your thoughts.
>> Kindest regards,
>> 
>> Eric
>> 
>> 
>> 
>> 
>> 
>> 
> 
> -- 
> Shadi Abou-Zahra - http://www.w3.org/People/shadi/
> Activity Lead, W3C/WAI International Program Office
> Evaluation and Repair Tools Working Group (ERT WG)
> Research and Development Working Group (RDWG)
>

Received on Thursday, 12 January 2012 23:07:29 UTC