Re: thoughts re: WCAG2.0 Techniques/Tests (my action item)?

Dear Tim and All,

At 15:51 12/07/2005, Tim Boland wrote:

>Per my "action item" from July 6 WCAG TTF teleconference, here are some 
>thoughts:
>
>
>(1) Perhaps it should be a goal that each technique should have a documented
>test assertion, derived from the technique's task, test files, and examples;
>this assertion would attempt to objectively define (measure) what it means for
>  a technique to be "successful" relative to the "related" WCAG2.0 success
>  criterion.  For more information on test assertions, please consult QA 
> SpecGL
>Good Practice 12 [1] .

I'm not convinced that techniques need test assertions.
There are two aspects in which the WCAG test suites differ from most other
W3C test suites:
1. We are always working with at least two specifications (WCAG + the
technology or technologies).
2. Test assertions can therefore expressed in two ways: as functional outcomes
or in a more technical, code-oriented way. I assume that the group prefers the
first type of assertion.
The techniques documents are not written to support the development of a test
suite but to help authors of web content. Defining tasks seems therefore
more important than defining test assertions. Test assertions are important
for test suites; for the techniques documents it is more important to assist
authors than to assist the development of test suites.


>(2) Perhaps it should be a goal that each technique should have documented
>(possibly as part of the test assertion or as an extension to it) exactly
>and specifically how such "success" as measured in (1) previous would 
>"satisfy"
>the relevant WCAG2.0 success criterion.  For example, for those techniques 
>"related"
>to WCAG2.0 G1.3L1SC1 [2], language could be included as "Structures within 
>the content are
>programmatically determined because...and then insert language "derived from"
>test assertion" of "related" technique(s)?

The tasks (when combined with the SC or SCs they relate to) seem currently the
best fit for this requirement.


>(3) Perhaps there should be a more "formal" process by which techniques are
>evaluated and approved for promoting WCAG2.0 conformance, to ensure not only
>that the requirements specified in Requirements for WCAG2.0 Checklists and 
>Techniques [3] are met, but also that conditions (1) and (2) mentioned 
>previously are
>satisfied (since the techniques will actually be "tested" to support 
>WCAG2.0 conformance).
>Perhaps any techniques that do not meet the previously-mentioned 
>requirements should
>be placed in a different category from those that do?

A description of a "Conformance Test Process for WCAG 2.0" is available at
http://www.w3.org/WAI/GL/WCAG20/tests/ctprocess.html. However, to make sure 
that
conditions (1) and (2) above are satisfied we may need a development 
process, not
just an evaluation & approval process. Is that what you have in mind?


>(4) Perhaps there should be consideration as to whether appropriate 
>"metadata" (with definitions
>for all applicable terms) could be used in managing baseline information 
>related to techniques
>(to attempt to manage the "complexity" related to baseline description?).
>A proposed QA Wiki test case metadata set of
>terms [4] might serve as illustration for this approach (since baseline is 
>related to techniques which is related to testing?);

These metadata are useful for test suites, but they would cause overload to 
a web developer
reading a techniques document.

>  perhaps some of these
>  terms are even applicable to baseline management as well.  I think there 
> may need to be a
>consistent methodology for managing the "complexity" of baseline 
>information in techniques.
>
>
>(5) Testing of accessibility issues implemented in a technology may have 
>different
>aspects than testing the basic features of the referenced technology 
>itself, and so I
>think these aspects may need to be distinguished in "testing" the 
>techniques.  For more
>information on testing per this point, please consult QA Test FAQ [5]  and 
>the QA Wiki topics on "Building a Test Suite [6].

The more I think about it, the more I have the impression that we're trying to
make the techniques documents serve two different purposes:
- providing guidance to web content developers ("techniques" in the usual 
sense);
- proving that the WCAG success criteria are testable (which is more 
relevant to
test suites).
If we want the techniques to be readable, they should be free of clutter 
such as
test case metadata [4]. If we want to prove that the WCAG success criteria are
testable, then we really need test cases (with test assertsion, test case 
metadata
and test files), but these would need to reference technology specific 
versions of
WCAG 2.0 rather than techniques, because the techniques don't contain test 
assertions
(they contain, and should contain, "tasks").

What do you think?

Regards,

Christophe Strobbe



>Thanks and best wishes
>Tim Boland NIST
>
>(...)
>[1]: http://www.w3.org/TR/2005/PR-qaframe-spec-20050629/#write-assertion-gp
>[2]: http://www.w3.org/TR/WCAG20/#content-structure-separation
>[3]: http://www.w3.org/TR/wcag2-tech-req/
>[4]: http://esw.w3.org/topic/TestCaseMetadata
>[5]: http://www.w3.org/QA/WG/2005/01/test-faq
>[6]: http://esw.w3.org/topic/QA
>

-- 
Christophe Strobbe
K.U.Leuven - Departement of Electrical Engineering - Research Group on 
Document Architectures
Kasteelpark Arenberg 10 - 3001 Leuven-Heverlee - BELGIUM
tel: +32 16 32 85 51
http://www.docarch.be/

Received on Monday, 1 August 2005 17:17:04 UTC