Confidence Claims - more discussion

Hi,

On today's ERT WG call we had some more discussion about the earl:confidence property and how we should be handling it in EARL. It seems we could agree on the following:

1. Even though the confidence claims are more related to test case descriptions than to the results, expressing them as part of the result is still an important aspect. Several tools are already using this property and would like to continue doing so.

2. earl:confidence is not simply a relay of the respective property in the test case description (testcase:confidence as a pseudo URI), but it is "processed" by the evaluation tool before it is inserted into a report. An example is to override the value in the test case description when a human evaluator executes the test.

3. earl:confidence should be based on a numeric value (such as percentage or interval). The values "high", "medium", and "low" should be mapped to appropriate numeric values but should remain available for describing ordinal values.

4. There will always be differences between tool results, also in earl:confidence. However, more clarity on how to assign confidence values will reduce the gap that is currently causing reduced interoperability of reports between tools.

5. It is unclear if the documentation on how to assign confidence is within the scope of EARL (as part of the EARL primer for example) or rather something relevant and important to have.


More input on how we should proceed with this property is very important at this stage.

Regards,
  Shadi


-- 
Shadi Abou-Zahra,       Web Accessibility Specialist for Europe 
World Wide Web Consortium (W3C),             http://www.w3.org/ 
Web Accessibility Initiative (WAI),      http://www.w3.org/WAI/ 
IST WAI-TIES Project (WAI-TIES)     http://www.w3.org/WAI/TIES/ 
Evaluation and Repair Tools (ERT WG), http://www.w3.org/WAI/ER/ 
2004, Route des Lucioles BP93 - 06560 Sophia-Antipolis - France 
Voice: +33(0)4 92 38 50 64             Fax: +33(0)4 92 38 78 22

Received on Tuesday, 31 May 2005 18:46:44 UTC