W3C home > Mailing lists > Public > public-wai-evaltf@w3.org > September 2012

Thoughts about the differentiation of reporting types (Basic | Detailed | In-depth)

From: <detlev.fischer@testkreis.de>
Date: Wed, 5 Sep 2012 14:47:21 +0200 (CEST)
To: public-wai-evaltf@w3.org
Message-Id: <20120905124721.9230368210EC@dd24924.kasserver.com>
Hi all,

Going through the new draft of WCAG-EM, I began to wonder whether we should differentiate Basic Report and Detailed Report a bit more, leading to thoughts whether we really need all three types, and in what sense they differ.

Here is the text quoted from 3.5.1 Step 5.a: Provide Documentation for Each Step:

* Basic Report
    "Only captures the successes and failures in meeting WCAG 2.0 conformance requirements globally for the entire website . For each WCAG 2.0 Success Criterion applicable as per 3.1.3 Step 1.c. Define the Conformance Target , the report identifies if it is met or not met in the selected sample of web pages . Where failures in meeting WCAG 2.0 Success Criteria are identified, each web page in which such a failure has been identified must be indicated in the report."

* Detailed Report
    "Captures the successes and failures in meeting WCAG 2.0 conformance requirements for each web page . For each WCAG 2.0 Success Criterion applicable as per 3.1.3 Step 1.c. Define the Conformance Target , the report identifies if it is met or not met in each web page in the selected sample of web pages . Where failures in meeting WCAG 2.0 Success Criteria on a web page are identified, each identified occurrence of such a failure must be indicated in the report."

It seems that while Basic Report lists SC and for each of these, failed pages, the Detailed Report lists sampled pages and for each of these, failed SC. The real difference is the requirement in the Detailed Report to list *each instance*, which could easily get very tedious and counter-productive (just think of a longish text using <br> instead of <p>). Would you really want a list of all these violations of SC 1.3.1 in a Detailed Report? Maybe we should change to "each identified occurrence of such a failure, or each type of failure where occuring instances are repetitive, must be indicated in the report."

The wording in  3.5.1 Step 5.a: to describe the types of report may be changed to acknowledge that the two ways of reporting are basically homologous:
* listing all pages (with failures) in the sample, then listing all failed SC per page
* listing all WCAG SC on the chosen level of conformance, then listing all pages that failed

I think we should not mandate either way of sorting results in Basic Report and Detailed Report; both can be mapped on the respective other or changed through some DB sort command in an evaluation tool. The way it reads now, it appears as if providing SC first, then failed pages (Basic Report) is something quite different from providing pages first, then SCs that failed (Detailed Report).  

The *real* difference between the two types of report is the requirement to enlist *all* the failure instances in the Detailed Report. If this is not done in a mechanical way (e.g., by providing line numbers, which BTW may still not be sufficiently accurate), it requires a comment identifying where and how a SC failed - which is getting close to the In-Depth report. 

The only added value I can currently see in the Detailed report is getting some rough quantitative measure of the number of instances where a failure occured (which can be quite a misleading indicator without referencing criticality of the failure).

Regards,
Detlev
--
Detlev Fischer
testkreis c/o feld.wald.wiese
Borselstraße 3-7 (im Hof), 22765 Hamburg

Mobil +49 (0)1577 170 73 84
Tel +49 (0)40 439 10 68-3
Fax +49 (0)40 439 10 68-5

http://www.testkreis.de
Beratung, Tests und Schulungen für barrierefreie Websites
Received on Wednesday, 5 September 2012 12:47:51 GMT

This archive was generated by hypermail 2.3.1 : Friday, 8 March 2013 15:52:15 GMT