W3C

Requirements for "Techniques for Automated and Semi-Automated Evaluation Tools"

This document provides a set of initial requirements that need to be incorporated in the document "Techniques for Automated and Semi-Automated Evaluation Tools". Further refinements of this document will occur under the scope of the Evaluation and Repair Tools Working Group (ERT WG) discussions.

This version:
http://www.w3.org/WAI/ER/WD-AERT/ED-requirements20130723
Previous version:
http://www.w3.org/WAI/ER/WD-AERT/ED-requirements20130701
Editor:
Carlos A Velasco, Fraunhofer Institute for Applied Information Technology FIT

Purpose of the document

The document presented here gathers requirements for the document "Techniques for Automated and Semi-Automated Evaluation Tools", in the following called the document. This requirements' document will present also some scenarios on the use of the main document.

The purpose of the document "Techniques for Automated and Semi-Automated Evaluation Tools" is to present typical features of web accessibility evaluation tools that will support the reader in defining different tool profiles.

Objectives of the document

The objectives of the document "Techniques for Automated and Semi-Automated Evaluation Tools" include the following:

  1. Describe to developers of web accessibility evaluation tools typical features of such tools and briefly present different perspectives on these features (examples of such features are listed in the section "Typical features of a web accessibility evaluation tool").
  2. Describe typical profiles for web accessibility evaluation tools according to different combinations of the aforementioned features.
  3. Support developers of web accessibility evaluation tools to understand the different types of web accessibility tests: automatic, semiautomatic and manual.
  4. Support developers of web accessibility evaluation tools to understand how to use WCAG 2.0 success criteria, sufficient techniques, advisory techniques, and common failures for web accessibility testing.

In addition, the document may provide additional information on supporting developers of web accessibility evaluation tools to present test results to different audiences and how to integrate their tools into different development workflows.

Audience of the document

The document "Techniques for Automated and Semi-Automated Evaluation Tools" is targeted mainly to development managers and developers of web accessibility evaluation tools. Under this scope, we will not distinguish between commercial and open source developers, although there are use cases that could be more relevant to one group than to the other.

A secondary audience of this document are users of accessibility evaluation tools like accessibility experts or web developers.

Types of tools within the scope of the document

Examples of tools that are included are:

Typical features of a web accessibility evaluation tool

The document will contain descriptions of different features that are included in accessibility evaluation tools, which help to classify them and to identify their limitations. Typical examples include:

Scenarios

Here we will present two or more scenarios which can put in context the recommendations of the document.

John: a development manager

John is a development manager in a small software company creating testing tools for mobile and desktop web applications. Due to increasing demand from customers, the company is evaluating the possibility to extend the software to evaluate web accessibility. John consults the document to get a general overview of typical features from accessibility evaluation tools. He also gathers information about resources that helped him to understand the implications of this new functionality and how their existing tools will map into the profiles defined in the document. He creates a matrix to compare the existing characteristics from its tool with the features of accessibility tools. With the result of this comparison, he is able to estimate the effort necessary to implement the new features of the tool and create an implementation roadmap.

Marc: a system architect

Marc is a system architect working for the IT department of a medium sized company, which needs to introduce accessibility evaluation functions in their own intranet content generation tools. Marc is familiar with software specification techinques, but not so acquainted with the accessibility domain. He will use the document as an input to generate the tool specification, given that it defines desirable, expected or simply typical features of evaluation tools. Marc choose the profile that best fits the needs his company, and draws inspiration from it to get a requirements list and a set of use cases. Marc prioritizes a required subset of features and use cases, while leaving others as optionally supported. He then wireframes user interface prototypes that provide a sensible representation for the different actors that will use the tool. In this protoype, the document may serve as background on the different ways the results of accessibility tests may be presented.

Issues not covered in this document

The following issues are not covered in this document:

References

  1. Web Content Accessibility Guidelines (WCAG) 2.0
  2. Website Accessibility Conformance Evaluation Methodology 1.0
  3. Developer Guide for Evaluation and Report Language (EARL) 1.0
  4. UWEM, Unified Web Evaluation Methodology version 1.2
  5. Requirements for web developers and web commissioners in ubiquitous Web 2.0 design and development (January 2012)

Table of contents

What follows is a preliminary table of contents for the document:

  1. Abstract
  2. Status of this document
  3. Introduction
    1. Audience of this document
    2. Document conventions
    3. Complementary resources
  4. Typical features of an evaluation tool
  5. Example profiles of evaluation tools
  6. References
  7. Appendix A: Customising results to different audiences
  8. Appendix B: Integrating the evaluation procedure into the development testing workflows