W3C home > Mailing lists > Public > public-wai-evaltf@w3.org > June 2012

Re: difference between techniques and procedures

From: Shadi Abou-Zahra <shadi@w3.org>
Date: Fri, 08 Jun 2012 17:01:41 +0200
Message-ID: <4FD213D5.5060001@w3.org>
To: Richard Warren <richard.warren@userite.com>
CC: Eval TF <public-wai-evaltf@w3.org>
Hi Richard,

We are converging... ;)


On 8.6.2012 15:18, RichardWarren wrote:
> Dear Shadi and Eval Team,
>
> I am concerned that we are making things overly complex.
>
> Our task is to develop a procedure for checking compliance with the
> Guidelines.

Agreed.


> We are not validating the method/s used by the designer to meet the
> guidelines. - Yes it makes our job easier if the designer has used
> specified techniques etc.

Agreed.


> But the final test is "are there text alternatives for non-text
> content?", "Does it work properly when using a keyboard?", "Can
> users avoid and correct mistakes?" etc...

Side note: if these testing procedures are not vetted then we run into 
the risk of having individual interpretations of the requirements that 
may be wrong. Given your example, what is "properly work with keyboard" 
if you don't back it up with a formalized testing procedure?


> Now if I use the 'procedures' as listed by W3C to check for compliance
> with each guideline I can tell the designer that I have done so and that
> his/her site either complies with the guidelines, or does not. My report
> will tell the designer which guidelines are complied with and which not.
> Any third party can check my results using the same procedures.

OK, so we assume vetted testing procedures that some else can also run.

So far we kind of agree but there is the issue with using the testing 
procedures of the Techniques without the surrounding context which is 
often essential to running the testing procedure. In particular the 
"Applicability" and "User Agent and Assistive Technology Support Notes" 
sections of a Technique seem to be very relevant to running the tests.


> That is all I need to do in my role as an evaluator. (Step 4 of section
> 3 Evaluation Procedure)
>
> In my role as a consultant I clearly need to do a lot more by
> identifying why and where it is non-compliant and how the designer
> should address the problems etc..

Agreed.


> I therefore still believe that steps 1.e and 4b should relate to the
> procedures used by the evaluator to check for compliance.

As said above, you can get misleading or wrong results if you do not 
consider the context in which a test procedure needs to be run in.


> As for Failure Techniques, I agree that they could be better written,
> but they are more relevant to the designer and consultant (i.e.
> analysing why/how a failure happens) rather that to the evaluator.

Maybe the issue is the more narrative parts of a Technique (including 
Failure Technique), such as "Description", "Example", and such?

Also, to be sure, to my understand we are talking primarily about 
testing procedure *from* Failure Techniques. Even if they are not 
written that way, implicitly they are testing procedures to check for 
failures rather than for compliance. Correct?

Best,
   Shadi


> Regards
>
> Richard
>
> -----Original Message----- From: Shadi Abou-Zahra
> Sent: Friday, June 08, 2012 1:03 PM
> To: RichardWarren
> Cc: Eval TF
> Subject: Re: difference between techniques and procedures
>
> I think we should continue to reference the "Techniques" rather than the
> test procedures which are a sub-part of a technique. The other parts of
> a technique can also be relevant for evaluation, for example the
> "applicability" clauses that an evaluators needs to consider too.
>
> Also, I think that one of the issues is that WCAG 2.0 and supporting
> documents do not explain "Failure Techniques" ("Common Failures") [1]
> clearly enough. We probably need to describe these a little more.
>
> [1] <http://www.w3.org/TR/WCAG20-TECHS/failures.html>
>
> Best,
> Shadi
>
>
> On 2.6.2012 12:50, RichardWarren wrote:
>> In the light of our discussions on step 1e I think that Step 4b also
>> appears to be confusing. I think we are trying to say that we will
>> record the testing techniques as described at
>> http://www.w3.org/TR/WCAG20-TECHS/intro.html#intro_testing_techs .
>>
>> When W3C describes techniques in its techniques collection it refers
>> to these testing techniques as “procedures” at the end of each
>> document under the title “Tests”. See an example at
>> http://www.w3.org/TR/WCAG20-TECHS/G1.html#G1-tests
>>
>> We too should use the words test procedure when we refer to what we do
>> when we check for compliance. This would avoid confusion between the
>> techniques that the web-designer uses to create an accessible page and
>> the processes we, as evaluators, use to check that the page is
>> compliant (either by using a W3C technique, or some other technique).
>>
>> Richard
>

-- 
Shadi Abou-Zahra - http://www.w3.org/People/shadi/
Activity Lead, W3C/WAI International Program Office
Evaluation and Repair Tools Working Group (ERT WG)
Research and Development Working Group (RDWG)
Received on Friday, 8 June 2012 15:02:16 GMT

This archive was generated by hypermail 2.3.1 : Friday, 8 March 2013 15:52:14 GMT