Re: result type="foo", confidence, ...

On Thu, 14 Apr 2005 05:09:16 +1000, Nils Ulltveit-Moe <nils@u-moe.no>  
wrote:


> Most humans are not able to specify exactly how confident they are that
> an accessibility claim is indeed a real problem for accessibility. They
> may be able to give a rough indication, though.
[...]
> What I am trying to say, is that it is not always possible to abstract
> oneself away from uncertainties that are inherent in the problem or
> testcase that is being investigating, and in such cases it is better to
> have a model that includes the uncertainty than pretending that the
> uncertainty is not there.

I agree that it is hard to specify exactly, and this is why I have not  
been keen on it. Iam slowly coming around to the idea, if it can be  
modelled in such a way that interoperability isn't more or less impossible.

>> From what I have discussed here, I am actually getting more convinced
> that a confidence interval or similar is useful, and not only for
> automatic tests. If confidence interval was used also for manual tests
> then one would be able to get feedback on how real people perceived
> different accessibility problems to be, which in turn could be used by
> W3C to improve the WCAG checklists.
[...]
> My conclusion is that maybe the confidence interval should not be
> mandatory, but I think it should be optional in EARL. And it should be
> modelled as a probability; i.e. an integer between 0 and 1.

I think if we are going to have it, then it should be an n-ary object that  
actually includes somespecification of how it was measured. For example,  
spamassassin point scores can be used interoperably between mail clients  
(perhaps this is a good use case).

This would leave us with, in the simple case, modelling the result as a  
blank node with a type (rather than using it as a direct property of the  
Assertion), and in the complex case adding a compound description of the  
confidence we have in the result.

(We could do both, but that means that it we have to require systems to  
support two different RDF graphs for a result - more complexity than we  
need, I think, since we should be able to constrain the graph easily  
enough).

So you stay with, in the simple case

<earl:Assertion>
   <earl:result r:type="...fail"/>

(instead of <earl:result r:resource="...fail"/> )

and if you want to do confidence you have something like the example in  
the spec (although I suggest we change the model for that to make it more  
or less require to say what scheme you use - even if it is just to  
identify that this is "Chaals' gut feeling scheme" :-)

cheers

Chaals

Received on Thursday, 14 April 2005 00:55:03 UTC