potential misuse of warnings (was Re: [ACTION] Warnings in EARL)

Hi,

As per my action item from this week's teleconference discussion [1], 
please find a scenario for potential misuse of the warnings as proposed 
by CarlosI [2]:

[1] <http://www.w3.org/2006/12/06-er-minutes#item04>
[2] <http://lists.w3.org/Archives/Public/public-wai-ert/2006Dec/0003>

The scenario proposed by CarlosI highlights the following three separate 
tests:

* CSS document validity
* background/foreground color contrast
* redefinition of properties

If the question is "does the document validate to CSS" then the answer 
is yes, and the RDF as presented by CarlosI seems sane (the two latter 
tests are only informative). However, if the question is "are there 
accessibility issues in the CSS" then the answer is no because the 
second test is actually a potential accessibility issue.

So the usage of the warning really becomes dependent on the context of 
the question and more importantly on the understanding of the tool 
developers. This means that a potential misuse of the warnings could be 
as follows:


<earl:TestResult rdf:ID="result">
   <earl:validity rdf:resource="&earl;#pass"/>
   <dc:title xml:lang="en">WCAG AA</dc:title>
   <dc:description rdf:parseType="Literal" xml:lang="en">
     <div xmlns="http://www.w3.org/1999/xhtml">
       <p>This document passed WCAG AA!</p>
     </div>
   </dc:description>
   <earl:warning rdf:resource="#warning1"/>
   <earl:warning rdf:resource="#warning2"/>
</earl:TestResult>

<earl:Warning rdf:ID="warning1">
   <dc:title xml:lang="en">User Check 1</dc:title>
   <dc:description rdf:parseType="Literal" xml:lang="en">
     <div xmlns="http://www.w3.org/1999/xhtml">
       <p>Manual evaluation needed to check ...</p>
     </div>
   </dc:description>
   <earl:instance rdf:resource="#instance1"/>
   <earl:instance rdf:resource="#instance2"/>
</earl:Warning>

<earl:Warning rdf:ID="warning2">
   <dc:title xml:lang="en">User Check 2</dc:title>
   <dc:description rdf:parseType="Literal" xml:lang="en">
     <div xmlns="http://www.w3.org/1999/xhtml">
       <p>Manual evaluation needed to check ...</p>
     </div>
   </dc:description>
   <earl:instance rdf:resource="#instanceN"/>
</earl:Warning>


Such a misuse is quite likely as some tool developers may prefer to show 
"pass" to their users rather than "failed" or "can not tell"; and the 
warning class as is would give them an easy opportunity to do so.

As a solution I propose the following:

1- If a warning is truly informative, then it is only additional 
information to the result. Thus it does not need additional pointers to 
a different location and it should only be a simple string.

2- Other types of more substantial warnings could be done by relating 
test results to each other. We've had discussion about this in the past 
and I am aware it is a can of worms but it may make sense.

Here is a concrete example of what I have in mind:


<earl:TestResult rdf:ID="result">
   <earl:validity rdf:resource="&earl;#pass"/>
   <dc:title xml:lang="en">LONGDESC Supplied</dc:title>
   <dc:description rdf:parseType="Literal" xml:lang="en">
     <div xmlns="http://www.w3.org/1999/xhtml">
       <p>LONGDESC attribute has been supplied.</p>
     </div>
   </dc:description>
   <earl:instance rdf:resource="#instance1"/>
   <earl:instance rdf:resource="#instance2"/>
   <earl:warning rdf:type="Literal">LONGDESC is not supported by many 
browsers</earl:warning>
   <earl:relatedResult rdf:resource="#result2"/>
</earl:TestResult>

<earl:TestResult rdf:ID="result2">
   <earl:validity rdf:resource="&earl;#cannotTell"/>
   <dc:title xml:lang="en">Check Link</dc:title>
   <dc:description rdf:parseType="Literal" xml:lang="en">
     <div xmlns="http://www.w3.org/1999/xhtml">
       <p>Manual evaluation of the link target needed.</p>
     </div>
   </dc:description>
   <earl:instance rdf:resource="#instance1"/>
   <earl:instance rdf:resource="#instance2"/>
   <earl:instance rdf:resource="#instanceN"/>
</earl:TestResult>


As mentioned, the problem with the "relatedResult" approach is that it 
is in turn a hidden test description formalism. This in turn reminds me 
of the "evidence" and "methodology" discussions and not wanting to 
revive these. Any suggestions to bypass this issue?

Other comments or feedback on the proposal as a whole?

Regards,
   Shadi


-- 
Shadi Abou-Zahra     Web Accessibility Specialist for Europe |
Chair & Staff Contact for the Evaluation and Repair Tools WG |
World Wide Web Consortium (W3C)           http://www.w3.org/ |
Web Accessibility Initiative (WAI),   http://www.w3.org/WAI/ |
WAI-TIES Project,                http://www.w3.org/WAI/TIES/ |
Evaluation and Repair Tools WG,    http://www.w3.org/WAI/ER/ |
2004, Route des Lucioles - 06560,  Sophia-Antipolis - France |
Voice: +33(0)4 92 38 50 64          Fax: +33(0)4 92 38 78 22 |

Received on Thursday, 7 December 2006 15:39:58 UTC