Test Considerations for ATAG2

Hi all,

I just realized I hadn't sent this to the list.

Basically, we need to create some supports for evaluators to help them determine if a tool meets ATAG 2.0 or not. Here is a start:


BEFORE YOU BEGIN:

Decide on a "Test Target Level"
===============================
To determine the Conformance Level of an authoring tool to ATAG 2.0, it is recommended that testers ensure Level A conformance, followed by Level AA, then finally Level AAA.

Proceeding in this way simplifies the construction and selection of test tools required.


Test Tools
==========
Test tools should reflect the "Test Target Level". So for example, the "Comprehensive non-accessible test content (Level A)" would only include level A errors, not AA or AAA.

Web Content Accessibility Test Procedure: 
-----------------------------------------
Procedure that is to be used whenever it is necessary to determine the accessibility of web content (e.g. in the authoring tool's own user interface or in the content produced by the tool). 
- The most complete test is to manually follow "How to Meet WCAG 2.0" (http://www.w3.org/WAI/WCAG20/quickref/)
- Some may find it useful to precede the manual test with an automated test. Free automated tools that accept URIs or pasted HTML code include:
 + WebAIM Wave
 + AChecker

Comprehensive accessible test content (Level A, AA, AAA)
--------------------------------------------------------
- a complete file (e.g. HTML) with no known accessibility problems to the given level.
- not needed if the tool does not import content or allow markup to be pasted in.

Comprehensive non-accessible test content (Level A, AA, AAA)
------------------------------------------------------------
- a complete file (e.g. HTML) with at least one accessibility issue for each WCAG 2.0 SC at the given level.
- not needed if the tool does not import content or allow markup to be pasted in.

(Array of) Atomic accessible test content (Level A, AA, AAA)
------------------------------------------------------------
- array of files (e.g. HTML) each containing elements that potentially could have accessibility issue at the given level, but which do not (e.g. an IMG with ALT text).
- not needed if the tool does not import content or allow markup to be pasted in.

(Array of) Atomic non-accessible test content (Level A, AA, AAA)
----------------------------------------------------------------
- array of files (e.g. HTML) each containing elements with one accessibility issue at the given level (e.g., IMG with no ALT text).
- not needed if the tool does not import content or allow markup to be pasted in.

(Array of) Content Pieces
---------------------------
- A selection of separate pieces of content that can be used as needed to test various SCs. 
- Should include: text passages, table content, images files, video files, audio files, etc. 



TEST CONSIDERATIONS ATTACHED TO EACH SC:

A.3.3.1 Test Considerations
===========================
http://www.w3.org/TR/2011/WD-ATAG20-20110721/#sc_a331

- This SC only applies to authoring tools that rendered visual time-based content such as video, animation, animated gifs etc.
- This SC does not cover non-editable previews, such as in a browser.
- The SC would cover web-based editing-views. In these case, pause and optional auto-play functions in the browser may be used to satisfy the SC.
- A tool will meet the "can be set to not play automatically" part of the SC if it never plays time-based content automatically or if there is a setting to turn off automatic playing on load.
- At tool will meet the SC "pause" part of the SC if, once playing starts there is a way to stop it, whether it is a pause or stop function.


B.1.1.1 Test Considerations
===========================
http://www.w3.org/TR/2011/WD-ATAG20-20110721/#sc_b111:

- This SC only applies to authoring tools that add anything to the end-user experience beyond what the author has experienced during the authoring session. Otherwise, the SC should be marked not applicable.
- The SC's note also makes it clear that the SC refers to developer-provided processes, not processes created by the author or other third-parties.
- If the auto-generation system primarily works by creating a wrapper around the author's entries, try to test the auto-generation system with as little author-entered content as possible. If author input is required, ensure that only accessible content is added and that all prompts are followed correctly.
- If the auto-generation system acts differently depending on the author's entries, try to test the auto-generation system with the "comprehensive accessible test content" (ed. Lots of features pre-checked for accessibility). If additional author input is required, ensure that only accessible content is added and that all prompts are followed correctly.
- Once the system produces output, follow the "Web Content Accessibility Test Procedure" (ed. Which we need to specify) to determine if it meets the WCAG 2.0 success criteria TARGET_LEVEL (ed. I think it is easier to test ATAG 2.0 with a TARGET_LEVEL in mind) 



Cheers,
Jan


-- 
(Mr) Jan Richards, M.Sc.
jrichards@ocad.ca | 416-977-6000 ext. 3957 | fax: 416-977-9844
Inclusive Design Research Centre (IDRC) | http://idrc.ocad.ca/
Faculty of Design | OCAD University

Received on Monday, 25 July 2011 19:48:43 UTC