Re: Confidence Claims - more discussion

Carlos Iglesias wrote:
> 
> 
>>So, could one say that the currently employed use case of the 
>>earl:confidence values is to express different levels of the 
>>pass/fail values? For example, in order to express the following:
>>
>>"This test failed for sure" -> validity=fail, confidence=high 
>>"This test probably passes" -> validity=pass, 
>>confidence=medium "This test may not be applicable" -> 
>>validity=NA, confidence=low
> 
> 
> I think this is the currently use, and, in this case, the mess is even
> greater:
> 
> What would be the barrier between "fail low" and "pass low"?
> Where would "cannotTell" fit? If you have a "pass low" or "fail low"
> confidence, is not the same as cannotTell?

OK, I'm going to have to find time to describe what Valet currently
does.  It avoids that problem.

>>If so, then do we want to continue doing that through the 
>>validity/confidence pair or do we rather want to introduce 
>>more granuality for validity (for example earl:ProbablyPasses 
>>as a subclass of earl:fail)?
> 
> 
> IMO it could be the good way:
> 
> Pass --> "pass high"
> ProblabyPass --> "pass medium"
> CannotTell --> "pass low" or "fail low"
> ProblabyFail --> "fail medium"
> Fail --> "fail high"

At the per-page level, that's what Valet gives you.  It's based on
an aggregation of all tests performed on the page and indicating
a possible fail.

Within a page, it's different.  The basic principle is, it's asserting
"This page [definitely|probably|maybe] (passes|fails), and here are the
details of things that [may] cause it to fail."  The details are the
results of the individual tests, each of which is associated with the
appropriate node in the markup.

But here we only need to record where a test has failed.  Take as an
example, misuse of <blockquote> for indentation.  Since this is very
widespread, Valet starts with a premise that any <blockquote> may be
a misuse.  But it also looks for a cite="..." attribute, on the grounds
that any blockquote with a cite is almost certainly being used
correctly.  If the blockquote has no cite, it is flagged as a fail
with medium confidence (the confidence level ascribed is inevitably
subjective here).  Crucially, if it has a cite, the test is passed,
and _nothing_ is recorded.  It is not really productive to load a
report full of tests that were passed!

General disclaimer: what I say about Valet is from memory:-)

-- 
Nick Kew

Received on Wednesday, 1 June 2005 20:18:00 UTC