ATAG2 test writing - Principle B.3

Useful definitions and procedures:
http://www.w3.org/WAI/AU/CR20/TestPrep.html

B.3.1.1 Checking Assistance (WCAG): If the authoring tool provides authors with the ability to add or modify web content in such a way that a WCAG 2.0 success criterion can be violated, then accessibility checking for that success criterion is provided (e.g., an HTML authoring tool that inserts images should check for alternative text; a video authoring tool with the ability to edit text tracks should check for captions). (Level A to meet WCAG 2.0 Level A success criteria; Level AA to meet WCAG 2.0 Level A and AA success criteria; Level AAA to meet all WCAG 2.0 success criteria)
- Note: Automated and semi-automated checking is possible (and encouraged) for many types of web content accessibility problems (WCAG). However, manual checking is the minimum requirement to meet this success criterion. In manual checking, the authoring tool provides authors with instructions for detecting problems, which authors must carry out by themselves. For more information on checking, see Implementing ATAG 2.0 - Appendix B: Levels of Checking Automation.

Test 0001 Assertion: Checking assistance (WCAG Level A) is present in less-restrictive authoring tools.
Test 0001 Author: Jan Richards
Test 0001 Steps:
1. Check whether the authoring tool allows authors to open/paste-in pre-existing content, open/paste-in "Comprehensive non-accessible test content (Level A)". If this is not possible, then select SKIP.
2. Search the user interface (and documentation) for accessibility checking functionality (which can be automated, semi-automated, manual instructions or a combination). If checking functionality is not present, then select FAIL.
3. Run the checker (if the checker is manual, this will entail following ALL checking instructions). If the checker provides detail about individual errors then you can open/paste-in all of the "Comprehensive non-accessible test content (Level A)" at once. If the checker provides only aggregate results, open/paste-in the "Comprehensive non-accessible test content (Level A)" one piece at a time. 
4. If the checker is able to cause all of the accessibility problems to be detected (automatically, semi-automatically, manually or a combination), then select PASS. Otherwise, select FAIL.

Test 0002 Assertion: Checking assistance (WCAG Level AA) is present in less-restrictive authoring tools.
Test 0002 Author: Jan Richards
Test 0002 Steps:
1. Check whether the authoring tool allows authors to open/paste-in pre-existing content, open/paste-in "Comprehensive non-accessible test content (Level AA)". If this is not possible, then select SKIP.
2. Search the user interface (and documentation) for accessibility checking functionality (which can be automated, semi-automated, manual instructions or a combination). If checking functionality is not present, then select FAIL.
3. Run the checker (if the checker is manual, this will entail following ALL checking instructions). If the checker provides detail about individual errors then you can open/paste-in all of the "Comprehensive non-accessible test content (Level AA)" at once. If the checker provides only aggregate results, open/paste-in the "Comprehensive non-accessible test content (Level A)" one piece at a time. 
4. If the checker is able to cause all of the accessibility problems to be detected (automatically, semi-automatically, manually or a combination), then select PASS. Otherwise, select FAIL.

Test 0003 Assertion: Checking assistance (WCAG Level AA) is present in less-restrictive authoring tools.
Test 0003 Author: Jan Richards
Test 0003 Steps:
1. Check whether the authoring tool allows authors to open/paste-in pre-existing content, open/paste-in "Comprehensive non-accessible test content (Level AAA)". If this is not possible, then select SKIP.
2. Search the user interface (and documentation) for accessibility checking functionality (which can be automated, semi-automated, manual instructions or a combination). If checking functionality is not present, then select FAIL.
3. Run the checker (if the checker is manual, this will entail following ALL checking instructions). If the checker provides detail about individual errors then you can open/paste-in all of the "Comprehensive non-accessible test content (Level AAA)" at once. If the checker provides only aggregate results, open/paste-in the "Comprehensive non-accessible test content (Level AAA)" one piece at a time. 
4. If the checker is able to cause all of the accessibility problems to be detected (automatically, semi-automatically, manually or a combination), then select PASS. Otherwise, select FAIL.

Test 0004 Assertion: Checking assistance (WCAG Level A) is present in more-restrictive authoring tools.
Test 0004 Author: Jan Richards
Test 0004 Steps:
1. Check whether the authoring tool DOES NOT allow authors to open/paste-in pre-existing content, open/paste-in "Comprehensive non-accessible test content (Level A)" and instead only allows authoring new content via its interface. If this is not the case, then select SKIP.
2. Using "Comprehensive non-accessible test content (Level A)" as a guide, attempt to create accessibility problems using the authoring tool. If this is not possible due to authoring restrictions (such as highly limited editing scope or compulsory fields), then select SKIP.
3. If accessibility problems have been added to the content, search the user interface (and documentation) for accessibility checking functionality (which can be automated, semi-automated, manual instructions or a combination). If checking functionality is not present, then select FAIL.
4. Run the checker (if the checker is manual, this will entail following ALL checking instructions). If the checker provides detail about individual errors then you can open/paste-in all of the "Comprehensive non-accessible test content (Level A)" at once. If the checker provides only aggregate results, open/paste-in the "Comprehensive non-accessible test content (Level A)" one piece at a time. 
5. If the checker is able to cause all of the accessibility problems to be detected (automatically, semi-automatically, manually or a combination), then select PASS. Otherwise, select FAIL.

Test 0005 Assertion: Checking assistance (WCAG Level AA) is present in more-restrictive authoring tools.
Test 0005 Author: Jan Richards
Test 0005 Steps:
1. Check whether the authoring tool DOES NOT allow authors to open/paste-in pre-existing content, open/paste-in "Comprehensive non-accessible test content (Level AA)" and instead only allows authoring new content via its interface. If this is not the case, then select SKIP.
2. Using "Comprehensive non-accessible test content (Level AA)" as a guide, attempt to create accessibility problems using the authoring tool. If this is not possible due to authoring restrictions (such as highly limited editing scope or compulsory fields), then select SKIP.
3. If accessibility problems have been added to the content, search the user interface (and documentation) for accessibility checking functionality (which can be automated, semi-automated, manual instructions or a combination). If checking functionality is not present, then select FAIL.
4. Run the checker (if the checker is manual, this will entail following ALL checking instructions). If the checker provides detail about individual errors then you can add all of the problems first and then run the checker. If the checker provides only aggregate results, you will need to add and then check for problems one at a time. 
5. If the checker is able to cause all of the accessibility problems to be detected (automatically, semi-automatically, manually or a combination), then select PASS. Otherwise, select FAIL.

Test 0006 Assertion: Checking assistance (WCAG Level A) is present in more-restrictive authoring tools.
Test 0006 Author: Jan Richards
Test 0006 Steps:
1. Check whether the authoring tool DOES NOT allow authors to open/paste-in pre-existing content, open/paste-in "Comprehensive non-accessible test content (Level AAA)" and instead only allows authoring new content via its interface. If this is not the case, then select SKIP.
2. Using "Comprehensive non-accessible test content (Level AAA)" as a guide, attempt to create accessibility problems using the authoring tool. If this is not possible due to authoring restrictions (such as highly limited editing scope or compulsory fields), then select SKIP.
3. If accessibility problems have been added to the content, search the user interface (and documentation) for accessibility checking functionality (which can be automated, semi-automated, manual instructions or a combination). If checking functionality is not present, then select FAIL.
4. Run the checker (if the checker is manual, this will entail following ALL checking instructions). If the checker provides detail about individual errors then you can add all of the problems first and then run the checker. If the checker provides only aggregate results, you will need to add and then check for problems one at a time. 
5. If the checker is able to cause all of the accessibility problems to be detected (automatically, semi-automatically, manually or a combination), then select PASS. Otherwise, select FAIL.

---

B.3.1.2 Help Authors Decide: If the authoring tool provides checks that require authors to decide whether a potential web content accessibility problem (WCAG) is correctly identified (i.e., manual checking and semi-automated checking), then instructions are provided from the check that describe how to decide. (Level A)

Test 0001 Assertion: Checking assistance (WCAG Level A) includes help to decide whether potential issues are actual issues, if user judgement is required.
Test 0001 Author: Jan Richards
Test 0001 Steps:
1. Examine the checking functionality identified in the WCAG Level A-related tests for B.3.1.1. If no checking functionality exists, then select SKIP.
2. If the checker automatically identifies all of the issues, the select SKIP.
3. If the checker relies on the author to verify found issues (as semi-automatic checks do) or find issues (as manual instructions do), then check whether support of some kind (e.g. instructions) is provided in each case to help the user make the required decision. If such support is provided, then select PASS, otherwise select FAIL.

Test 0002 Assertion: Checking assistance (WCAG Level AA) includes help to decide whether potential issues are actual issues, if user judgement is required.
Test 0002 Author: Jan Richards
Test 0002 Steps:
1. Examine the checking functionality identified in the WCAG Level AA-related tests for B.3.1.1. If no checking functionality exists, then select SKIP.
2. If the checker automatically identifies all of the issues, the select SKIP.
3. If the checker relies on the author to verify found issues (as semi-automatic checks do) or find issues (as manual instructions do), then check whether support of some kind (e.g. instructions) is provided in each case to help the user make the required decision. If such support is provided, then select PASS, otherwise select FAIL.

Test 0003 Assertion: Checking assistance (WCAG Level AAA) includes help to decide whether potential issues are actual issues, if user judgement is required.
Test 0003 Author: Jan Richards
Test 0003 Steps:
1. Examine the checking functionality identified in the WCAG Level AAA-related tests for B.3.1.1. If no checking functionality exists, then select SKIP.
2. If the checker automatically identifies all of the issues, the select SKIP.
3. If the checker relies on the author to verify found issues (as semi-automatic checks do) or find issues (as manual instructions do), then check whether support of some kind (e.g. instructions) is provided in each case to help the user make the required decision. If such support is provided, then select PASS, otherwise select FAIL.

---

B.3.1.3 Help Authors Locate: If the authoring tool provides checks that require authors to decide whether a potential web content accessibility problem (WCAG) is correctly identified (i.e., manual checking and semi-automated checking), then the relevant content is identified to the authors. (Level A)
- Note: Depending on the nature of the editing-view and the scope of the potential web content accessibility problem (WCAG), identification might involve highlighting elements or renderings of elements, displaying line numbers, or providing instructions.

Test 0001 Assertion: Checking assistance (WCAG Level A) includes help to locate potential issues if user judgement is required.
Test 0001 Author: Jan Richards
Test 0001 Steps:
1. Examine the checking functionality identified in the WCAG Level A-related tests for B.3.1.1. If no checking functionality exists, then select SKIP.
2. If the checker automatically identifies all of the issues, the select SKIP.
3. If the checker relies on the author to verify found issues (as semi-automatic checks do) or find issues (as manual instructions do), then check whether a mechanism of some kind (e.g. hyperlink to the location, identification by line number, instructions on how to locate, etc.) is provided in each case to help the user determine where the potential issue is located within the content. If such a mechanism is provided, then select PASS, otherwise select FAIL.

Test 0002 Assertion: Checking assistance (WCAG Level AA) includes help to locate potential issues if user judgement is required.
Test 0002 Author: Jan Richards
Test 0002 Steps:
1. Examine the checking functionality identified in the WCAG Level AA-related tests for B.3.1.1. If no checking functionality exists, then select SKIP.
2. If the checker automatically identifies all of the issues, the select SKIP.
3. If the checker relies on the author to verify found issues (as semi-automatic checks do) or find issues (as manual instructions do), then check whether a mechanism of some kind (e.g. hyperlink to the location, identification by line number, instructions on how to locate, etc.) is provided in each case to help the user determine where the potential issue is located within the content. If such a mechanism is provided, then select PASS, otherwise select FAIL.

Test 0003 Assertion: Checking assistance (WCAG Level AAA) includes help to locate potential issues if user judgement is required.
Test 0003 Author: Jan Richards
Test 0003 Steps:
1. Examine the checking functionality identified in the WCAG Level AAA-related tests for B.3.1.1. If no checking functionality exists, then select SKIP.
2. If the checker automatically identifies all of the issues, the select SKIP.
3. If the checker relies on the author to verify found issues (as semi-automatic checks do) or find issues (as manual instructions do), then check whether a mechanism of some kind (e.g. hyperlink to the location, identification by line number, instructions on how to locate, etc.) is provided in each case to help the user determine where the potential issue is located within the content. If such a mechanism is provided, then select PASS, otherwise select FAIL.

---

B.3.1.4 Status Report: If the authoring tool provides checks, then authors can receive an accessibility status report based on the results of the accessibility checks. (Level AA)
- Note: The format of the accessibility status report is not specified and they might include a listing of problems detected or a WCAG 2.0 conformance level, etc..

Test 0001 Assertion: Accessibility status reporting is present.
Test 0001 Author: Jan Richards
Test 0001 Steps:
1. Examine the checking functionality identified for B.3.1.1. If no checking functionality exists, then select SKIP.
2. If the checking functionality does not result in any information being returned to the author about accessibility issues, then select FAIL.
3. If the information returned is gathered together in some way (a conformance statement, a list of accessibility problems, etc.) , then select PASS.
4. If the information returned is scattered throughout the content (e.g. problems are underlined) AND there is no overall indication of status (e.g. a status bar icon indicating that problems exist), then select FAIL.

---

B.3.1.5 Programmatic Association of Results: If the authoring tool provides checks, then the authoring tool can programmatically associate accessibility checking results with the web content that was checked. (Level AA)

Test 0001 Assertion: Checking results can be programmatic associated to the content.
Test 0001 Author: Jan Richards
Test 0001 Steps:
1. Examine the checking functionality identified for B.3.1.1. If no checking functionality exists, then select SKIP.
2. Examine the checking functionality (including settings and documentation) for any functionality that allows the results of the checking to be programmatically associated with the content (e.g. a file containing the checking results that includes a hyperlink back to the checked content). 
3. If this functionality is present, then select PASS. Otherwise, select FAIL.

---

B.3.2.1 Repair Assistance (WCAG): If checking (see Success Criterion B.3.1.1) can detect that a WCAG 2.0 success criterion is not met, then repair suggestion(s) are provided: (Level A to meet WCAG 2.0 Level A success criteria; Level AA to meet WCAG 2.0 Level A and AA success criteria; Level AAA to meet all WCAG 2.0 success criteria)
- Note: Automated and semi-automated repair is possible (and encouraged) for many types of web content accessibility problems (WCAG). However, manual repair is the minimum requirement to meet this success criterion. In manual repair, the authoring tool provides authors with instructions for repairing problems, which authors must carry out by themselves. For more information on repair, see Implementing ATAG 2.0 - Appendix C: Levels of Repair Automation.

Test 0001 Assertion: Checking assistance (WCAG Level A) includes help to repair all detected issues.
Test 0001 Author: Jan Richards
Test 0001 Steps:
1. Examine the checking functionality identified in the WCAG Level A-related tests for B.3.1.1. If no checking functionality exists, then select SKIP.
2. If the authoring tool automatically repairs all identified issues, then select SKIP.
3. For each identified issue, check whether support of some kind (e.g. dialog with repair options, instructions on how to repair, etc.) is provided. 
4. If such support is provided for every identified issue, then select PASS, otherwise select FAIL.

Test 0002 Assertion: Checking assistance (WCAG Level AA) includes help to repair all detected issues.
Test 0002 Author: Jan Richards
Test 0002 Steps:
1. Examine the checking functionality identified in the WCAG Level AA-related tests for B.3.1.1. If no checking functionality exists, then select SKIP.
2. If the authoring tool automatically repairs all identified issues, then select SKIP.
3. For each identified issue, check whether support of some kind (e.g. dialog with repair options, instructions on how to repair, etc.) is provided. 
4. If such support is provided for every identified issue, then select PASS, otherwise select FAIL.

Test 0003 Assertion: Checking assistance (WCAG Level AAA) includes help to repair all detected issues.
Test 0003 Author: Jan Richards
Test 0003 Steps:
1. Examine the checking functionality identified in the WCAG Level AAA-related tests for B.3.1.1. If no checking functionality exists, then select SKIP.
2. If the authoring tool automatically repairs all identified issues, then select SKIP.
3. For each identified issue, check whether support of some kind (e.g. dialog with repair options, instructions on how to repair, etc.) is provided. 
4. If such support is provided for every identified issue, then select PASS, otherwise select FAIL.

Received on Thursday, 26 July 2012 15:24:59 UTC