W3C home > Mailing lists > Public > public-auto-wcag@w3.org > July 2015

Introducing the auto-wcag user input template [via Automated WCAG Monitoring Community Group]

From: W3C Community Development Team <team-community-process@w3.org>
Date: Fri, 24 Jul 2015 08:28:53 +0000
To: public-auto-wcag@w3.org
Message-ID: <cf2415dfe9361ab0d32edd6ca6abc1be@www.w3.org>
Web accessibility evaluations can serve a number of different purposes ranging
from  quality assurance and error repair, individual reports and awareness
raising, to  benchmarking and monitoring. Many people who would like to know
the accessibility status of a web page aren’t experts in the field. In such
situations they rely on tools that produce reports about (potential) errors.

Only some aspects of the Web Content Accessibility Guidelines (WCAG) 2.0 can be
checked automatically. The majority of Success Criteria require human judgment.

The W3C Automated WCAG Monitoring Community Group is developing a new approach 
to involve non-experts in the data collection process for an accessibility
study. By combining the benefits of automated and manual testing we aim to
improve both the quality and the coverage of evaluation results.
Automated checker tools and human judgment
Human intervention is needed in web accessibility testing because automatic
testing alone cannot cover all aspects of WCAG. Many of the tools mentioned on
the W3C Web Accessibility Evaluation Tools List acknowledge that fact and report
issues that can’t be tested automatically as “warnings”or “potential

The main target audience of these tools are web developers. The tools are
intended for use during the creation of a web site and for subsequent quality
assurance. This leads to some limitations: The output of the tools contains a
lot of technical terms such as HTML element names and references to technical
documentation such as the Techniques for WCAG 2.0. Therefore the tools can only
be used by persons with web development expertise.

Moreover, repair instructions like “Ensure that the img element’s alt text
serves the same purpose as the image.” are targeted towards improvements of
the web content, and are not appropriate in the context of monitoring and status

The WCAG Evaluation Methodology recommends involving users with disabilities in
the evaluation of a web site. However, if this is done in an informal way
without controlled setting, the results are often biased because personal
opinion, individual expertise, or other factors influence the result. Especially
the level of expertise of the user has a strong influence on the accuracy of the

This leads to the conclusion that evaluators should be grouped by their level of
expertise rather than by type of disability. Clearly worded questions could lead
to better answers from all users with little knowledge of web accessibility.
Structured semi-automatic evaluation approach
The objective of auto-wcag is to create a process with clear structure and
instructions that  are easy to understand so that even non-experts can follow.
Standardized questions reduce the influence of individual opinion. A clear
wording and predefined answer options instead of general statements or repair
instructions lead to higher quality answers and thus to more reliable results.

Each auto-wcag test case consists of a selector and one or more test steps.
There are automatic steps, which can be done by a tool, and manual steps, which
require human input.

The manual steps describe tool support and instructions for non-expert users.
Tool support can include highlighting the test subject, presenting alternative
content that is not directly visible without special settings in the user agent,
or providing other specific presentations of the content. These features allow
the users to focus on the test subject. The users don’t have to identify the
relevant item on the page and the distraction caused by irrelevant items it

Clear instructions and additional help text enable non-experts to answer the
questions as well. The template also captures two additional properties of the
test steps: the requirement of interaction and the consideration of context.
Check description
The original content and the (programmatically determined) alternative content
are presented alongside each other. The question asks if the alternative
describes the original content. This type applies to all kinds of non-text
content such as images, audio and video as covered by Success Critereon 1.1.1
Non-text Content and Guideline 1.2 Time-based Media. For example, a paragraph of
text is presented together with the programmatically determined language, the
user is asked if the language is specified correctly.
Check presentation
The web content (or parts of it) is presented to the user in a specific way. For
instance with resized text or in linearized form. The questions address features
and problems of this presentation. This type applies for example to 1.4.4 Resize
text: The text of the web page is resized to 200% and the user is asked if all
content is still present.
Check interaction
The complete web page is presented to the users. In this type of test the user
is instructed to interact with the web content and to make a statement about the
operability. This type is used to check Success Criteria addressing operability.
Moreover the behavior of focus, input, and error handling can be covered. For
example, the user is asked to move the focus around the web page with the
keyboard and to answer the question if the focus got trapped in any component of
the page.
Manual selector
So far we have covered semi-automatic tests where the tool can determine
applicability and present the preprocessed subject of the test to the non-expert
user. However, there are also cases where applicability cannot be determined
automatically and the user acts as a manual selector. In this type of user input
the user is asked to identify content items that might cause accessibility
barriers, such as use of color or other sensory characteristics to reference
elements of the web page. It can also be applied to instances of flashing and
auto-updating content that can’t be controlled by the user. For example, users
could be asked to identify moving, blinking, or scrolling content that plays
automatically and cannot be paused.
Next steps
Some participants of the auto-wcag community group are currently implementing
the prototype of a User Testing Tool based on the questions developed in the
structured approach described in this post. The tool runs in the user’s web
browser and connects to a database storing the user input. The data can then be
combined with the results from other (automatic) tools to create a report about
the evaluated web content.
About the author
Annika Nietzio is a web accessibility expert working at Research Instistute
Technologie and Disability in Germany. In the EIII project she is exploring new
ways to combine the results from automated and manual web accessibility


This post sent on Automated WCAG Monitoring Community Group

'Introducing the auto-wcag user input template'


Learn more about the Automated WCAG Monitoring Community Group: 

Received on Friday, 24 July 2015 08:28:56 UTC

This archive was generated by hypermail 2.3.1 : Friday, 24 July 2015 08:28:57 UTC