Re: Thoughts about the differentiation of reporting types (Basic | Detailed | In-depth)

Hi Detlev,

Good observation in my opinion. So it is good to have these discussions. Please continue this discussion thread.

For sampling we may also want to discuss with WCAG WG during TPAC in Lyon. Maybe we have to cut the sampling discussion into smaller chunks. Just not sure how to do that. 

In the meantime, don't forget to review the new documents so we can decide if this is ok for asking public comments.
Kindest regards,

Eric

Op 5 sep. 2012 om 18:36 heeft "Detlev Fischer" <detlev.fischer@testkreis.de> het volgende geschreven:

> Hi Eric,
> 
> this was not meant to hold up publication of the draft, just an observation I have thrown on the list - I see it as a relatively minor issue. I do not mind opening an issue for that though.
> 
> More important (but after publication) seems to me the discussion of an approach for sampling processes to which I have already supplied a draft. But I trust that will be discussed in due time. I would be really curious for feedback by others on the list - does it fit your needs in evaluating web applications? Is it too prescriptive? Missing something important? etc.
> 
> Best,
> Detlev
> 
> On 5 Sep 2012, at 15:53, Velleman, Eric wrote:
> 
>> Hi Detlev,
>> 
>> Good input! We added this section because of the need defined by EvalTF for different levels of reporting and the possible impact on the evaluation (time) effort. I applaud your proposal to rethink this section. But I would like to start that after we publish the current version as a Public Working Draft. Hope that is ok?
>> 
>> Would you then be willing to propose an alternative setup for these goals that we can discuss on the list and Telco (but after publication of the current version)? We will then have time to discuss that while the Public Review period is on.
>> 
>> If EvalTF is ok and you are ok with it, I can log it as task in the tracker.
>> Kindest regards,
>> 
>> Eric
>> 
>> ________________________________________
>> Van: detlev.fischer@testkreis.de [detlev.fischer@testkreis.de]
>> Verzonden: woensdag 5 september 2012 14:47
>> Aan: public-wai-evaltf@w3.org
>> Onderwerp: Thoughts about the differentiation of reporting types (Basic | Detailed |  In-depth)
>> 
>> Hi all,
>> 
>> Going through the new draft of WCAG-EM, I began to wonder whether we should differentiate Basic Report and Detailed Report a bit more, leading to thoughts whether we really need all three types, and in what sense they differ.
>> 
>> Here is the text quoted from 3.5.1 Step 5.a: Provide Documentation for Each Step:
>> 
>> * Basic Report
>>   "Only captures the successes and failures in meeting WCAG 2.0 conformance requirements globally for the entire website . For each WCAG 2.0 Success Criterion applicable as per 3.1.3 Step 1.c. Define the Conformance Target , the report identifies if it is met or not met in the selected sample of web pages . Where failures in meeting WCAG 2.0 Success Criteria are identified, each web page in which such a failure has been identified must be indicated in the report."
>> 
>> * Detailed Report
>>   "Captures the successes and failures in meeting WCAG 2.0 conformance requirements for each web page . For each WCAG 2.0 Success Criterion applicable as per 3.1.3 Step 1.c. Define the Conformance Target , the report identifies if it is met or not met in each web page in the selected sample of web pages . Where failures in meeting WCAG 2.0 Success Criteria on a web page are identified, each identified occurrence of such a failure must be indicated in the report."
>> 
>> It seems that while Basic Report lists SC and for each of these, failed pages, the Detailed Report lists sampled pages and for each of these, failed SC. The real difference is the requirement in the Detailed Report to list *each instance*, which could easily get very tedious and counter-productive (just think of a longish text using <br> instead of <p>). Would you really want a list of all these violations of SC 1.3.1 in a Detailed Report? Maybe we should change to "each identified occurrence of such a failure, or each type of failure where occuring instances are repetitive, must be indicated in the report."
>> 
>> The wording in  3.5.1 Step 5.a: to describe the types of report may be changed to acknowledge that the two ways of reporting are basically homologous:
>> * listing all pages (with failures) in the sample, then listing all failed SC per page
>> * listing all WCAG SC on the chosen level of conformance, then listing all pages that failed
>> 
>> I think we should not mandate either way of sorting results in Basic Report and Detailed Report; both can be mapped on the respective other or changed through some DB sort command in an evaluation tool. The way it reads now, it appears as if providing SC first, then failed pages (Basic Report) is something quite different from providing pages first, then SCs that failed (Detailed Report).
>> 
>> The *real* difference between the two types of report is the requirement to enlist *all* the failure instances in the Detailed Report. If this is not done in a mechanical way (e.g., by providing line numbers, which BTW may still not be sufficiently accurate), it requires a comment identifying where and how a SC failed - which is getting close to the In-Depth report.
>> 
>> The only added value I can currently see in the Detailed report is getting some rough quantitative measure of the number of instances where a failure occured (which can be quite a misleading indicator without referencing criticality of the failure).
>> 
>> Regards,
>> Detlev
>> --
>> Detlev Fischer
>> testkreis c/o feld.wald.wiese
>> Borselstraße 3-7 (im Hof), 22765 Hamburg
>> 
>> Mobil +49 (0)1577 170 73 84
>> Tel +49 (0)40 439 10 68-3
>> Fax +49 (0)40 439 10 68-5
>> 
>> http://www.testkreis.de
>> Beratung, Tests und Schulungen für barrierefreie Websites
>> 
>> 
>> 
>> 
>> 
> 
> -- 
> Detlev Fischer
> testkreis - das Accessibility-Team von feld.wald.wiese
> c/o feld.wald.wiese
> Thedestraße 2
> 22767 Hamburg
> 
> Tel   +49 (0)40 439 10 68-3
> Mobil +49 (0)1577 170 73 84
> Fax   +49 (0)40 439 10 68-5
> 
> http://www.testkreis.de
> Beratung, Tests und Schulungen für barrierefreie Websites
> 
> 
> 
> 
>

Received on Wednesday, 5 September 2012 16:51:05 UTC