Re: Re: difference between techniques and procedures

Hi all,

Maybe it helps to stick to the terms established in WCAG as much as possible:

* WCAG Sufficient Techniques: Every test in a Sufficient Technique states that failing the test does not necessarily mean that the SC has not been met in some other way. So that should take care of the misunderstandings mentioned by Kerstin. However, Sufficient Techniques are called sufficient for a reason: *if* they are used (and often they are, even if presented in a more abstract bare-bones way than encountered in the wild) they should *usually* be sufficient to satisfy the SC because they usually show sufficient accessibility support for a broad user base (public internet). Still, there may be Failures encountered in testing that show that the SC is in fact not satisfied. So any test procedure trundling through options for Sufficient Techniques also needs to check if there are documented Failures attached to the SC that may apply to the content under test.

* WCAG Failures: these come without the disclaimer that the SC might be met in some other way. That means, in principle, if a failure is found to apply, the SC fails. I think it is probably not helpful to talk about 'Failure Techniques'. There can be web developer techniques that disregard WCAG SC and therefore fail, but WCAG Failures are certainly not meant to be applied by web developers. They are more clearly aimed at an Evaluation context, helping evaluators to determine if a violation of WCAG SCs is present. What we need to discuss is if *every* failure regardles of criticality automatically fails the SC, or if there are edge cases. Think of F32 where a single word spaced with spaces instead of CSS would fail a page that in all other respects satisfies SC 1.3.2.

* Testing processes, test procedures, etc: I think we should refer to tests in WCAG Techniques and Failures in a way that makes clear that we refer to the test section in these Techniques and Failures (e.g., "WCAG Technique test" or "WCAG Failure test") . Thes tests are not necessarily the same as a test procedure in an evaluation. I have pointed out before that there is a lot of redundancy in WCAG Technique tests, so chances are that for reasons of efficiency, testers in the field won't repeat every redundant test step in every possibly applicable WCAG Technique. Take img alternative texts as an example. The evaluation procedure exposes the alt attributes (e.g,  IE > WAT toolbar > "List images") , compares each alt text with the respective image and its purpose (linked teaser, illustraton, decorative graphic, etc) and then determines, in line with or based on the documented techniques, whether the purpose or content of the image is adequately described (in the alt attribute 
 itself, or in linked, included or fallback context). That is what most people do in practice, whatever tool they use for that. 
We have had different opinions here as to the value of documenting evaluation procedures that aggregate atomic tests. One risk of that is that novel techniques may be missed. That topic will surely come up again in discussions.

Regards,
Detlev


--
Detlev Fischer
testkreis c/o feld.wald.wiese
Borselstraße 3-7 (im Hof), 22765 Hamburg

Mobil +49 (0)1577 170 73 84
Tel +49 (0)40 439 10 68-3
Fax +49 (0)40 439 10 68-5

http://www.testkreis.de
Beratung, Tests und Schulungen für barrierefreie Websites



----- Original Message -----
From: k.probiesch@googlemail.com
To: shadi@w3.org
Date: 13.06.2012 09:09:32
Subject: Re: difference between techniques and procedures


> Hi Shadi, all,
> 
> I don't think that it is a problem to use the term "testing procedures" because this is actually what an evaluator is doing.
> 
> A lot of discussions about the relevance of the techniques has shown that there are many misunderstandings and that many developers (and not only developers) are thinking that it is just allowed to use techniques which are in the techniques document. If we use ' techniques' even as 'optional' I fear that we will bring in more confusion in this discussion and I think it is likely that some take the template     but use techniques or much worse an own sample of preferred techniques as subsections.
> 
> Best
> 
> Kerstin
> 
> Via Mobile
> 
> Am 12.06.2012 um 23:51 schrieb Shadi Abou-Zahra <shadi@w3.org>:
> 
>> Hi Mike,
>> 
>> Yes, it has been a lengthy thread with different points raised.
>> 
>> As to your comment, I don't think it is a good idea to coin new terms. WCAG already differentiates between "Techniques" and "General Failures".
>> 
>> I think we are trying to fix the wrong problem...
>> 
>> Best,
>>  Shadi
>> 
>> 
>> On 12.6.2012 21:30, Michael S Elledge wrote:
>>> Hi All--
>>> 
>>> This has been a long and exhaustive (exhausting?) thread. I think
>>> referring to our evaluations as "procedures" and developer methods as
>>> "techniques" will help to reduce confusion. I also agree that it is
>>> important to remember that ultimately our role as evaluators is to
>>> determine whether a client has met the success criteria established by
>>> the W3C in WCAG 2.0. While it may add value to point a client to a
>>> better technique than he/she is using, especially if their technique has
>>> failed, it is not necessary to accomplishing our goal as evaluator.
>>> 
>>> Mike
>>> 
>>> On 6/8/2012 8:19 PM, Vivienne CONWAY wrote:
>>>> Hi Richard and all
>>>> I've been following the discussion, but sadly was too snowed under
>>>> (and that's really hard in Australia!) until this morning to respond.
>>>> 
>>>> I think Richard that you capture my thoughts very concisely. In
>>>> reality what methods the designer uses to meet the guidelines isn't
>>>> really the issue for us as evaluators. We need to look at what methods
>>>> he's used to see if he's found something different that works, but
>>>> mostly we are checking to see that whatever techniques he uses meets
>>>> the guidelines. As you said, if there are images, they must have
>>>> meaningful alternatives and conform with the guidelines for 1.1.1 .I
>>>> think we should, cite the failures - particularly if the designer has
>>>> clearly used a documented failure technique in the design.
>>>> 
>>>> In our reporting, we obviously can talk about what different
>>>> techniques the designer used to meet the guidelines and what we did to
>>>> test those features. But we do need to document carefully the
>>>> procedures we used for testing. In my reporting, I also describe why
>>>> certain things the designer has done are not compliant, and provide
>>>> suggestions for things he could do that would remedy this.
>>>> 
>>>> I think in the end if we reference our EM, that will cover a lot of
>>>> the 'how' we tested part. I also think I'd like to see more work done
>>>> on the Failure Techniques. I think that would help a lot of designers
>>>> - in my mind that is who these are primarily written for. They help us
>>>> in that we can point the designer to the Failure Technique so they can
>>>> see the documentation of why a certain technique is not acceptable.
>>>> 
>>>> Hope that all makes sense?
>>>> 
>>>> 
>>>> 
>>>> 
>>>> Regards
>>>> 
>>>> Vivienne L. Conway, B.IT(Hons), MACS CT, AALIA(cs)
>>>> PhD Candidate& Sessional Lecturer, Edith Cowan University, Perth, W.A.
>>>> Director, Web Key IT Pty Ltd.
>>>> v.conway@ecu.edu.au
>>>> v.conway@webkeyit.com
>>>> Mob: 0415 383 673
>>>> 
>>>> This email is confidential and intended only for the use of the
>>>> individual or entity named above. If you are not the intended
>>>> recipient, you are notified that any dissemination, distribution or
>>>> copying of this email is strictly prohibited. If you have received
>>>> this email in error, please notify me immediately by return email or
>>>> telephone and destroy the original message.
>>>> ________________________________________
>>>> From: RichardWarren [richard.warren@userite.com]
>>>> Sent: Friday, 8 June 2012 9:18 PM
>>>> To: Shadi Abou-Zahra
>>>> Cc: Eval TF
>>>> Subject: Re: difference between techniques and procedures
>>>> 
>>>> Dear Shadi and Eval Team,
>>>> 
>>>> I am concerned that we are making things overly complex.
>>>> 
>>>> Our task is to develop a procedure for checking compliance with the
>>>> Guidelines.
>>>> 
>>>> We are not validating the method/s used by the designer to meet the
>>>> guidelines. - Yes it makes our job easier if the designer has used
>>>> specified
>>>> techniques etc. But the final test is "are there text alternatives for
>>>> non-text content?", "Does it work properly when using a keyboard?", "Can
>>>> users avoid and correct mistakes?" etc...
>>>> 
>>>> Now if I use the 'procedures' as listed by W3C to check for compliance
>>>> with
>>>> each guideline I can tell the designer that I have done so and that
>>>> his/her
>>>> site either complies with the guidelines, or does not. My report will
>>>> tell
>>>> the designer which guidelines are complied with and which not. Any third
>>>> party can check my results using the same procedures.
>>>> 
>>>> That is all I need to do in my role as an evaluator. (Step 4 of section 3
>>>> Evaluation Procedure)
>>>> 
>>>> In my role as a consultant I clearly need to do a lot more by identifying
>>>> why and where it is non-compliant and how the designer should address the
>>>> problems etc..
>>>> 
>>>> I therefore still believe that steps 1.e and 4b should relate to the
>>>> procedures used by the evaluator to check for compliance.
>>>> 
>>>> As for Failure Techniques, I agree that they could be better written, but
>>>> they are more relevant to the designer and consultant (i.e. analysing
>>>> why/how a failure happens) rather that to the evaluator.
>>>> 
>>>> Regards
>>>> 
>>>> Richard
>>>> 
>>>> -----Original Message-----
>>>> From: Shadi Abou-Zahra
>>>> Sent: Friday, June 08, 2012 1:03 PM
>>>> To: RichardWarren
>>>> Cc: Eval TF
>>>> Subject: Re: difference between techniques and procedures
>>>> 
>>>> I think we should continue to reference the "Techniques" rather than the
>>>> test procedures which are a sub-part of a technique. The other parts of
>>>> a technique can also be relevant for evaluation, for example the
>>>> "applicability" clauses that an evaluators needs to consider too.
>>>> 
>>>> Also, I think that one of the issues is that WCAG 2.0 and supporting
>>>> documents do not explain "Failure Techniques" ("Common Failures") [1]
>>>> clearly enough. We probably need to describe these a little more.
>>>> 
>>>> [1]<http://www.w3.org/TR/WCAG20-TECHS/failures.html>
>>>> 
>>>> Best,
>>>> Shadi
>>>> 
>>>> 
>>>> On 2.6.2012 12:50, RichardWarren wrote:
>>>>> In the light of our discussions on step 1e I think that Step 4b also
>>>>> appears to be confusing. I think we are trying to say that we will
>>>>> record
>>>>> the testing techniques as described at
>>>>> http://www.w3.org/TR/WCAG20-TECHS/intro.html#intro_testing_techs .
>>>>> 
>>>>> When W3C describes techniques in its techniques collection it refers to
>>>>> these testing techniques as “procedures” at the end of each document
>>>>> under
>>>>> the title “Tests”. See an example at
>>>>> http://www.w3.org/TR/WCAG20-TECHS/G1.html#G1-tests
>>>>> 
>>>>> We too should use the words test procedure when we refer to what we do
>>>>> when we check for compliance. This would avoid confusion between the
>>>>> techniques that the web-designer uses to create an accessible page
>>>>> and the
>>>>> processes we, as evaluators, use to check that the page is compliant
>>>>> (either by using a W3C technique, or some other technique).
>>>>> 
>>>>> Richard
>>>> --
>>>> Shadi Abou-Zahra - http://www.w3.org/People/shadi/
>>>> Activity Lead, W3C/WAI International Program Office
>>>> Evaluation and Repair Tools Working Group (ERT WG)
>>>> Research and Development Working Group (RDWG)
>>>> 
>>>> This e-mail is confidential. If you are not the intended recipient you
>>>> must not disclose or use the information contained within. If you have
>>>> received it in error please return it to the sender via reply e-mail
>>>> and delete any record of it from your system. The information
>>>> contained within is not the opinion of Edith Cowan University in
>>>> general and the University accepts no liability for the accuracy of
>>>> the information provided.
>>>> 
>>>> CRICOS IPC 00279B
>>>> 
>>>> 
>>>> 
>>> 
>> 
>> -- 
>> Shadi Abou-Zahra - http://www.w3.org/People/shadi/
>> Activity Lead, W3C/WAI International Program Office
>> Evaluation and Repair Tools Working Group (ERT WG)
>> Research and Development Working Group (RDWG)
>>

Received on Wednesday, 13 June 2012 08:06:08 UTC