W3C home > Mailing lists > Public > public-wai-evaltf@w3.org > September 2011

Re: Requirements draft - objectivity

From: Denis Boudreau <dboudreau@accessibiliteweb.com>
Date: Sun, 18 Sep 2011 22:21:43 -0400
Message-id: <7C0DD63C-4E58-4008-9850-7673B526016B@accessibiliteweb.com>
To: Eval TF <public-wai-evaltf@w3.org>
Hello again,

On 2011-09-15, at 1:39 AM, Detlev Fischer wrote:

> Quoting Kerstin Probiesch <k.probiesch@googlemail.com>:
>> Do we want that a tester can manipulate the results?
> DF: of course not, but this cannot be ensured by objectivity (whatever that would mean in practice) but only by some measure of quality control: a second tester or independent verification of results (also, verification of the adequacy of the page sample)

DB: I also support the idea of a methodology that supposes a round of quality control assured by a second evaluator going over the work of the first one to make sure it follows the proper way to do things.

>> I don't mean the case that something was overlooked but the case that something was willingly overlooked. Or the other Way round.
> DF: Well, if someone wants to distort results there will probably always ways to do that, I would not start from that assumption. Is one imperfect or missing alt attributes TRUE or FALSE for SC 1.1.1 applied to the entire page? What about a less than perfect heading structure? etc, etc. There is, "objectively", always leeway, room for interpretation, and I think we unfortunately DO need agreement with reference to cases / examples that set out a model for how they should be rated.

DB: +1.

>> If not we need Objectivity as a Requirement. Just Agreement on something is not enough.
> DF: Can you explain what in your view the requirement of "objectivity" should entail *in practice*, as part of the test procedure the methodology defines?

DB: "Agreement" on something should imply some ways to document what those "acceptable ways" to do/audit page elements mean.

>> And again: No Objectivity - no standardized methodology.

DB: Looks to me we'll need definitions to make sure we're all talking about the same thing we we talk about words like objectivity and agreement.

Received on Monday, 19 September 2011 02:22:15 UTC

This archive was generated by hypermail 2.3.1 : Tuesday, 6 January 2015 20:40:18 UTC