W3C home > Mailing lists > Public > public-wai-evaltf@w3.org > September 2012

Re: Thoughts about the differentiation of reporting types (Basic | Detailed | In-depth)

From: Detlev Fischer <detlev.fischer@testkreis.de>
Date: Wed, 5 Sep 2012 18:47:57 +0200
Cc: "public-wai-evaltf@w3.org" <public-wai-evaltf@w3.org>
Message-Id: <A50C0171-265F-467E-B1DC-E1C7C737E02B@testkreis.de>
To: "Velleman, Eric" <evelleman@bartimeus.nl>
Hi Eric,

this was not meant to hold up publication of the draft, just an  
observation I have thrown on the list - I see it as a relatively minor  
issue. I do not mind opening an issue for that though.

More important (but after publication) seems to me the discussion of  
an approach for sampling processes to which I have already supplied a  
draft. But I trust that will be discussed in due time. I would be  
really curious for feedback by others on the list - does it fit your  
needs in evaluating web applications? Is it too prescriptive? Missing  
something important? etc.

Best,
Detlev

On 5 Sep 2012, at 15:53, Velleman, Eric wrote:

> Hi Detlev,
>
> Good input! We added this section because of the need defined by  
> EvalTF for different levels of reporting and the possible impact on  
> the evaluation (time) effort. I applaud your proposal to rethink  
> this section. But I would like to start that after we publish the  
> current version as a Public Working Draft. Hope that is ok?
>
> Would you then be willing to propose an alternative setup for these  
> goals that we can discuss on the list and Telco (but after  
> publication of the current version)? We will then have time to  
> discuss that while the Public Review period is on.
>
> If EvalTF is ok and you are ok with it, I can log it as task in the  
> tracker.
> Kindest regards,
>
> Eric
>
> ________________________________________
> Van: detlev.fischer@testkreis.de [detlev.fischer@testkreis.de]
> Verzonden: woensdag 5 september 2012 14:47
> Aan: public-wai-evaltf@w3.org
> Onderwerp: Thoughts about the differentiation of reporting types  
> (Basic | Detailed |  In-depth)
>
> Hi all,
>
> Going through the new draft of WCAG-EM, I began to wonder whether we  
> should differentiate Basic Report and Detailed Report a bit more,  
> leading to thoughts whether we really need all three types, and in  
> what sense they differ.
>
> Here is the text quoted from 3.5.1 Step 5.a: Provide Documentation  
> for Each Step:
>
> * Basic Report
>    "Only captures the successes and failures in meeting WCAG 2.0  
> conformance requirements globally for the entire website . For each  
> WCAG 2.0 Success Criterion applicable as per 3.1.3 Step 1.c. Define  
> the Conformance Target , the report identifies if it is met or not  
> met in the selected sample of web pages . Where failures in meeting  
> WCAG 2.0 Success Criteria are identified, each web page in which  
> such a failure has been identified must be indicated in the report."
>
> * Detailed Report
>    "Captures the successes and failures in meeting WCAG 2.0  
> conformance requirements for each web page . For each WCAG 2.0  
> Success Criterion applicable as per 3.1.3 Step 1.c. Define the  
> Conformance Target , the report identifies if it is met or not met  
> in each web page in the selected sample of web pages . Where  
> failures in meeting WCAG 2.0 Success Criteria on a web page are  
> identified, each identified occurrence of such a failure must be  
> indicated in the report."
>
> It seems that while Basic Report lists SC and for each of these,  
> failed pages, the Detailed Report lists sampled pages and for each  
> of these, failed SC. The real difference is the requirement in the  
> Detailed Report to list *each instance*, which could easily get very  
> tedious and counter-productive (just think of a longish text using  
> <br> instead of <p>). Would you really want a list of all these  
> violations of SC 1.3.1 in a Detailed Report? Maybe we should change  
> to "each identified occurrence of such a failure, or each type of  
> failure where occuring instances are repetitive, must be indicated  
> in the report."
>
> The wording in  3.5.1 Step 5.a: to describe the types of report may  
> be changed to acknowledge that the two ways of reporting are  
> basically homologous:
> * listing all pages (with failures) in the sample, then listing all  
> failed SC per page
> * listing all WCAG SC on the chosen level of conformance, then  
> listing all pages that failed
>
> I think we should not mandate either way of sorting results in Basic  
> Report and Detailed Report; both can be mapped on the respective  
> other or changed through some DB sort command in an evaluation tool.  
> The way it reads now, it appears as if providing SC first, then  
> failed pages (Basic Report) is something quite different from  
> providing pages first, then SCs that failed (Detailed Report).
>
> The *real* difference between the two types of report is the  
> requirement to enlist *all* the failure instances in the Detailed  
> Report. If this is not done in a mechanical way (e.g., by providing  
> line numbers, which BTW may still not be sufficiently accurate), it  
> requires a comment identifying where and how a SC failed - which is  
> getting close to the In-Depth report.
>
> The only added value I can currently see in the Detailed report is  
> getting some rough quantitative measure of the number of instances  
> where a failure occured (which can be quite a misleading indicator  
> without referencing criticality of the failure).
>
> Regards,
> Detlev
> --
> Detlev Fischer
> testkreis c/o feld.wald.wiese
> Borselstraße 3-7 (im Hof), 22765 Hamburg
>
> Mobil +49 (0)1577 170 73 84
> Tel +49 (0)40 439 10 68-3
> Fax +49 (0)40 439 10 68-5
>
> http://www.testkreis.de
> Beratung, Tests und Schulungen für barrierefreie Websites
>
>
>
>
>

-- 
Detlev Fischer
testkreis - das Accessibility-Team von feld.wald.wiese
c/o feld.wald.wiese
Thedestraße 2
22767 Hamburg

Tel   +49 (0)40 439 10 68-3
Mobil +49 (0)1577 170 73 84
Fax   +49 (0)40 439 10 68-5

http://www.testkreis.de
Beratung, Tests und Schulungen für barrierefreie Websites
Received on Wednesday, 5 September 2012 16:36:22 GMT

This archive was generated by hypermail 2.3.1 : Friday, 8 March 2013 15:52:15 GMT