Re: EARL Guideline Pass/Fail Confidence

Hi,

I don't think confidence is a particularly accurate measure. In some cases we
are saying "cannotTell" but adding a suspicion, in some cases we are almost
certain (some people seem always to be certain :-).

I have no objection to people using a confidence scale, but I suspect that we
should look at the use cases and whether we  can say something else more
useful.

(See also my recent email about the bug in my intro re using rdf:resource
when it should be rdf:type or something even more complex...)

Cheers

Chaals

On Thu, 15 Jan 2004, Chris Ridpath wrote:

>
>Charles has an example of EARL that shows how to express that a page
>passes/fails an accessibility guideline. It's listed in his Coding EARL (for
>non experts) document at:
>http://www.w3.org/2001/sw/Europe/talks/200311-earl/all.htm
>
>The EARL code looks like:
>
>  <earl:Assertion>
>    <earl:subject rdf:resource="#http://www.w3.org/" />
>    <earl:result
>rdf:resource="http://www.w3.org/WAI/ER/EARL/nmg-strawman#Pass"/>
>    <earl:testcase
>rdf:resource="http://example.org/1999/xhtml#transitional"/>
>    <earl:assertedBy rdf:resource="http://validator.w3.org" />
>    <earl:mode
>rdf:resource="http://www.w3.org/WAI/ER/EARL/nmg-strawman#automatic"/>
>    <earl:message>This page is valid XHTML</earl:message>
>  </earl:Assertion>
>
>Would this be a better assertion if there was an added 'confidence'
>statement? Example:
>
><earl:confidence
>rdf:resource="http://www.w3.org/WAI/ER/EARL/nmg-strawman#high" />
>or
><earl:confidence rdf:resource=http://www.w3.org/WAI/ER/EARL/nmg-strawman#low
>/>
>
>An automated checker tool can only detect some problems, not all. It's up to
>a person to determine if the page passes all accessibility checks. For
>example, only a person can determine if an image does/doesn't require a long
>description.
>
>If the EARL expressed that the guideline was passed with a 'high' confidence
>then it would mean that all accessibility checks had passed - machine and
>human. If the confidence was 'low' then it would mean that only checks that
>are machine testable had passed - one or more checks that require human
>intervention had not passed.
>
>Using the confidence statement an automated checking tool could tell the
>user that "likely the page will pass but you still need a human to make some
>accessibility checks".
>
>Chris
>
>
>

Charles McCathieNevile  http://www.w3.org/People/Charles  tel: +61 409 134 136
SWAD-E http://www.w3.org/2001/sw/Europe         fax(france): +33 4 92 38 78 22
 Post:   21 Mitchell street, FOOTSCRAY Vic 3011, Australia    or
 W3C, 2004 Route des Lucioles, 06902 Sophia Antipolis Cedex, France

Received on Monday, 26 January 2004 07:33:00 UTC