Re: Requirements draft - objectivity

Hello again,


On 2011-09-15, at 1:39 AM, Detlev Fischer wrote:

> Quoting Kerstin Probiesch <k.probiesch@googlemail.com>:
> 
>> Do we want that a tester can manipulate the results?
> 
> DF: of course not, but this cannot be ensured by objectivity (whatever that would mean in practice) but only by some measure of quality control: a second tester or independent verification of results (also, verification of the adequacy of the page sample)

DB: I also support the idea of a methodology that supposes a round of quality control assured by a second evaluator going over the work of the first one to make sure it follows the proper way to do things.


>> I don't mean the case that something was overlooked but the case that something was willingly overlooked. Or the other Way round.
> 
> DF: Well, if someone wants to distort results there will probably always ways to do that, I would not start from that assumption. Is one imperfect or missing alt attributes TRUE or FALSE for SC 1.1.1 applied to the entire page? What about a less than perfect heading structure? etc, etc. There is, "objectively", always leeway, room for interpretation, and I think we unfortunately DO need agreement with reference to cases / examples that set out a model for how they should be rated.

DB: +1.


>> If not we need Objectivity as a Requirement. Just Agreement on something is not enough.
> 
> DF: Can you explain what in your view the requirement of "objectivity" should entail *in practice*, as part of the test procedure the methodology defines?

DB: "Agreement" on something should imply some ways to document what those "acceptable ways" to do/audit page elements mean.


>> And again: No Objectivity - no standardized methodology.

DB: Looks to me we'll need definitions to make sure we're all talking about the same thing we we talk about words like objectivity and agreement.

/Denis

Received on Monday, 19 September 2011 02:22:15 UTC