RE: What is a subject of a test?

 

> -----Mensaje original-----
> De: Charles McCathieNevile [mailto:charles@sidar.org] 
> Enviado el: jueves, 31 de marzo de 2005 17:11
> Para: Carlos Iglesias; shadi@w3.org; public-wai-ert@w3.org
> Asunto: Re: What is a subject of a test?
> 
> On Thu, 31 Mar 2005 20:54:17 +1000, Carlos Iglesias 
> <carlos.iglesias@fundacionctic.org> wrote:
> 
> > Another example of ambiguity in the specification:
> >
> > Instances of ValidityLevel
> >
> >     * cannotTell
> >     * fail
> >     * notApplicable
> >     * notTested
> >     * pass
> >
> > When to use each one?
> >
> > Let's continue with this example:
> >
> > Checkpoint 5.3: Do not use tables for layout unless the table makes 
> > sense when linearized. Otherwise, if the table does not make sense, 
> > provide an alternative equivalent (which may be a  
> linearized version).
> > [Priority 2]
> >
> > If the tool is checking a web page and it detects that it has no 
> > tables, which will be the validity level for this ckeckpoint?
> >
> > A- notApplicable
> >
> >> From the tool's point of view there are no tables and the 
> checkpoint 
> >> is
> > not applicable because there is nothing to check
> >
> > B- pass
> >
> > But an accessibility expert could think "it's not using 
> tables, so it 
> > passes the chekpoint"
> 
> Having read the WCAG spec a lot of times, it seems that a 
> properly designed tool and an expert evaluator will get the 
> same results nearly all the time... (of course a badly 
> designed tool and a person testing without much idea of how 
> to do it will not :-)


I absolutely disagree. If another evaluator gets different results in an evaluation, I prefer to think that it's due the implicit ambiguity of the WCAG spec, and different persons could have different opinions, instead thinking that it's due to the use of a bad tool or a possible precarious knowledge of the evaluator.


> > What I mean is that if we don't have a clear specification then it 
> > will be open to personal interpretation.
> 
> Sure. But this is irrelevant to the EARL spec - it is a 
> question of how good a particular spec we are testing against is.

So, do you think that is the spec we are testing that should explicitly when a test case fail, pass, is not tested, not applicable, etc. ?
I think that EARL could have something to do with this. For example:

ValidityLevel
 
Pass - When the test case is tested and passed
NotApplicable - When there is no subject to be tested
...

> In practical terms this particular ambiguity isn't such a big 
> deal anyway.  
> Mot specs are pretty clear on whether a result of Not 
> Applicable is equivalent to a pass in determining overall 
> conformance (as in the case of WCAG specs) or not...

In practical terms most specs are pretty clear, not all.


 
--------------------------------------

Carlos Iglesias Moro

Fundación CTIC
Parque Científico-Tecnológico de Gijón
33203 - Gijón, Asturias, España

teléfono: +34 984291212
fax: +34 984390612
email: carlos.iglesias@fundacionctic.org
URL: http://www.fundacionctic.org

Received on Friday, 1 April 2005 13:06:41 UTC