more discussion on warnings in EARL

Hi,

I'm wondering whether the concept of "warning" (as we have been 
discussing it) really belongs in EARL. During the last face-to-face we 
had decided that EARL only records the *outcome* of tests and not 
additional information about the relationship of the tests. However, 
issuing a warning seems to require knowledge about how the tests relate.

For simple and purely informative warnings (such as "LONGDESC not 
supported by many browsers"), it may be handy to record this directly in 
EARL. However, as we get to more complex types of warnings (such as the 
CSS validation example in [1]), it starts to get into the application 
level and somewhat messy. Below are two types of warning to outline how 
these could be addressed outside EARL:


#1. WARNINGS THROUGH TESTING WORKFLOW
<http://www.w3.org/TR/2006/WD-mobileOK-basic10-tests-20061113/#test_auto_refresh>
- The testing framework, in this example the draft MobilOK Test 3.1, 
defines a clear sequence for carrying out four atomic tests. Each result 
for each of these four tests could be recorded in an EARL assertion. The 
overall outcome of Test 3.1 depends on how these four atomic tests are 
combined. Also when a warning is issued is defined by the framework.
- In this case, it doesn't make sense for the tool to issue a warning 
when the test "If a Refresh HTTP header is present" is positive. The 
warning may or may not be issued depending on the outcome of the next 
test to be carried out. This also applies to the CSS examples brought up 
in earlier threads on the topic of warnings in EARL.
- Conclusion: the warning in this case is not on the level of EARL but 
comes from the higher level testing framework.


#2. WARNINGS THROUGH TEST DESCRIPTIONS
<http://www.w3.org/TR/2006/WD-WCAG20-TECHS-20060427/#H45>
- The test description, in this example the WCAG 2.0 Technique H45, 
describes the applicability of the test. For example that "Some older 
assistive technologies do not support the longdesc attribute. Supported 
in leading Assistive technologies now." This type of information may be 
important for the developer and/or evaluator and may need to be 
communicated in a different way.
- In this case, the tool could issue a warning as soon as it carries out 
this test. The warning is purely informative and supplements the outcome 
of the test. However, the text of this warning (that the tool issues to 
the end user) is already in the test description and does not have to be 
recorded in EARL as well.
- Conclusion: the warning in this case *could* be stored in EARL but it 
should also come from the higher level test description.


So again, we currently have the following three options on what to do 
with warnings in EARL:

A. drop the idea of warnings in EARL completely
B. adopt a simple literal earl:warning property
C. adopt a more comprehensive earl:Warning class

Other options? Preferences?

Regards,
   Shadi
[1] <http://lists.w3.org/Archives/Public/public-wai-ert/2006Dec/0003>


-- 
Shadi Abou-Zahra     Web Accessibility Specialist for Europe |
Chair & Staff Contact for the Evaluation and Repair Tools WG |
World Wide Web Consortium (W3C)           http://www.w3.org/ |
Web Accessibility Initiative (WAI),   http://www.w3.org/WAI/ |
WAI-TIES Project,                http://www.w3.org/WAI/TIES/ |
Evaluation and Repair Tools WG,    http://www.w3.org/WAI/ER/ |
2004, Route des Lucioles - 06560,  Sophia-Antipolis - France |
Voice: +33(0)4 92 38 50 64          Fax: +33(0)4 92 38 78 22 |

Received on Thursday, 14 December 2006 11:33:59 UTC