- From: Denis Boudreau <dboudreau@accessibiliteweb.com>
- Date: Sun, 18 Sep 2011 22:21:43 -0400
- To: Eval TF <public-wai-evaltf@w3.org>
Hello again, On 2011-09-15, at 1:39 AM, Detlev Fischer wrote: > Quoting Kerstin Probiesch <k.probiesch@googlemail.com>: > >> Do we want that a tester can manipulate the results? > > DF: of course not, but this cannot be ensured by objectivity (whatever that would mean in practice) but only by some measure of quality control: a second tester or independent verification of results (also, verification of the adequacy of the page sample) DB: I also support the idea of a methodology that supposes a round of quality control assured by a second evaluator going over the work of the first one to make sure it follows the proper way to do things. >> I don't mean the case that something was overlooked but the case that something was willingly overlooked. Or the other Way round. > > DF: Well, if someone wants to distort results there will probably always ways to do that, I would not start from that assumption. Is one imperfect or missing alt attributes TRUE or FALSE for SC 1.1.1 applied to the entire page? What about a less than perfect heading structure? etc, etc. There is, "objectively", always leeway, room for interpretation, and I think we unfortunately DO need agreement with reference to cases / examples that set out a model for how they should be rated. DB: +1. >> If not we need Objectivity as a Requirement. Just Agreement on something is not enough. > > DF: Can you explain what in your view the requirement of "objectivity" should entail *in practice*, as part of the test procedure the methodology defines? DB: "Agreement" on something should imply some ways to document what those "acceptable ways" to do/audit page elements mean. >> And again: No Objectivity - no standardized methodology. DB: Looks to me we'll need definitions to make sure we're all talking about the same thing we we talk about words like objectivity and agreement. /Denis
Received on Monday, 19 September 2011 02:22:15 UTC