Re: Towards a description of ATAG 2.0's approach to testing

Hi all,,

We discussed this message today:
http://lists.w3.org/Archives/Public/w3c-wai-au/2012JulSep/0041.html

As a result of the conversation, I propose am updating as follows (with @@ to mark things we didn't decide on):

1. Testing for ATAG 2.0 conformance can include numerous steps.
- The tests will be performable by independent testers although it is understood that developers, with access to their own development tools, will have an efficiency advantage in performing this testing.
- @@Statistical sampling techniques may be used when the number of user interface screens exceeds 5. 
 
2. ATAG 2.0 should be possible on a variety of OS platforms, but the precise details of how to do this differ by platform and change over time.
- The tests will not specify precisely how to make an application accessible on each particular platform. Instead, the tests will specify that the evaluator must have a "Platform Accessibility Service Test Procedure" ready before they start. It is expected that a brief description of the procedure will be provided, @@e.g. "A combination of automated and manual testing of the user interface and its communication with the accessibility API. Spot-checking with assistive technologies was also performed."

3. ATAG 2.0 often refers to WCAG 2.0 as the recommendation that authoring tools and authors should be seeking to meet in the produced content (and with the authoring interface). However, WCAG 2.0 itself points to its WCAG 2.0 Techniques for implementation guidance with respect to particular formats, but with the important proviso that techniques are non-normative. Furthermore, not all formats have WCAG 2.0 Techniques yet.
- The tests will not specify precisely how to meet WCAG 2.0. Instead, the tests will specify that the evaluator must have a " Web Content Accessibility Test Procedure" ready before they start. It is expected that a brief description of the procedure will be provided, @@e.g. "HTML content was tested with a combination of automated testing using the AChecker online tool and manual testing.  The WCAG 2.0 Techniques informed the testing, especially the 'sufficient techniques' and the 'common failures'. HTML Spot-checking with a screen reader was also performed."

4. As with any ATAG 2.0 SCs are written in absolute language, but in some cases well-implemented products may include (a) bugs or (b) pockets of little used functionality that have not yet been updated for accessibility.
- @@For CR, the group may choose to exclude parts of a tool if the WG judges that the parts are sufficiently separate from the main flow of the tool (e.g. an image editor plugin for an HTML editor).

5. There is some probability of issues being overlooked during testing for CR.
- @@The group will assume that CR evaluations are done in good faith, but don't represent guarantees.

Cheers,
Jan

(MR) JAN RICHARDS
PROJECT MANAGER
INCLUSIVE DESIGN RESEARCH CENTRE (IDRC)
OCAD UNIVERSITY

T 416 977 6000 x3957
F 416 977 9844
E jrichards@ocadu.ca

Received on Monday, 26 November 2012 22:16:32 UTC