W3C home > Mailing lists > Public > public-wai-evaltf@w3.org > September 2011

Re: Requirements draft - objectivity

From: Detlev Fischer <fischer@dias.de>
Date: Thu, 15 Sep 2011 07:39:34 +0200
Message-ID: <20110915073934.148417t48hld29w6@webmail.dias.de>
To: public-wai-evaltf@w3.org
Quoting Kerstin Probiesch <k.probiesch@googlemail.com>:

> Central question:
> Do we want that a tester can manipulate the results?

DF: of course not, but this cannot be ensured by objectivity (whatever  
that would mean in practice) but only by some measure of quality  
control: a second tester or independent verification of results (also,  
verification of the adequacy of the page sample)
> I don't mean the case that something was overlooked but the case  
> that something was willingly overlooked. Or the other Way round.

DF: Well, if someone wants to distort results there will probably  
always ways to do that, I would not start from that assumption. Is one  
imperfect or missing alt attributes TRUE or FALSE for SC 1.1.1 applied  
to the entire page? What about a less than perfect heading structure?  
etc, etc. There is, "objectively", always leeway, room for  
interpretation, and I think we unfortunately DO need agreement with  
reference to cases / examples that set out a model for how they should  
be rated.
> If not we need Objectivity as a Requirement. Just Agreement on  
> something is not enough.

DF: Can you explain what in your view the requirement of "objectivity"  
should entail *in practice*, as part of the test procedure the  
methodology defines?

> And again: No Objectivity - no standardized methodology.
> Kerstin
> Via Mobile
> Am 14.09.2011 um 12:09 schrieb Detlev Fischer <fischer@dias.de>:
>> DF: Just one point on objective, objectivity:
>> This is not an easy concept - it relies on a proof protocol. For  
>> example, you would *map* a page instance tested to a documented  
>> inventory of model cases to establish how you should rate it  
>> against a particular SC. Often this is easy, but there are many  
>> "not ideal" cases to be dealt with.
>> So "objective" sounds nice but it does not remove the problem that  
>> there will be cases that do not fit the protocol, at which point a  
>> human (or group, community) will have to make an informed mapping  
>> decision or extend the protocol to include the new instance. I  
>> think "agreed interpretation" hits it nicely because there is the  
>> community element in it which is quite central to WCAG 2.0 (think  
>> of defining accessibility support)
>> Regards,
>> Detlev
>>> Comment (KP): I understand the Denis' arguments. The more I think about
>>> this: neither "unique interpretation" nor "agreed interpretation" work very
>>> well. I would like to suggest "Objective". Because of the following reason:
>>> It would be one of Criteria for the quality of tests and includes Execution
>>> objectivity, Analysis objectivity and Interpretation objectivity.  
>>> If we will
>>> have in some cases 100% percent fine, if not we can discuss the  
>>> "tolerance".
>>> I would suggest:
>>> (VC)  I'm still contemplating this one.  I can see both arguments  
>>> as plausible.
>>> I'm okay with 'objectivity' but think it needs more explanation  
>>> i.e. who defines
>>> how objective it is?

Detlev Fischer PhD
DIAS GmbH - Daten, Informationssysteme und Analysen im Sozialen
Geschäftsführung: Thomas Lilienthal, Michael Zapp

Telefon: +49-40-43 18 75-25
Mobile: +49-157 7-170 73 84
Fax: +49-40-43 18 75-19
E-Mail: fischer@dias.de

Anschrift: Schulterblatt 36, D-20357 Hamburg
Amtsgericht Hamburg HRB 58 167
Geschäftsführer: Thomas Lilienthal, Michael Zapp
Received on Thursday, 15 September 2011 05:39:56 UTC

This archive was generated by hypermail 2.3.1 : Tuesday, 6 January 2015 20:40:18 UTC