W3C home > Mailing lists > Public > public-wai-evaltf@w3.org > February 2012

Draft for 5.3 Procedure for evaluation

From: Detlev Fischer <fischer@dias.de>
Date: Thu, 16 Feb 2012 10:20:47 +0100
Message-ID: <4F3CCA6F.2060303@dias.de>
To: EVAL TF <public-wai-evaltf@w3.org>
Dear EVAL TF,

Here is an attempt to provide a draft for section 5.3 "Procedure for 
evaluation" of our methodology.

The current version
http://www.w3.org/WAI/ER/conformance/ED-methodology-20120209
has the following draft text:

"This subclause provides a step by step description of the evaluation of 
the website (sample). This does not include going into the guidelines, 
success criteria etc from WCAG 2.0 or related techniques. It could be 
possible to propose different ways to evaluate the guidelines: one by 
one, per theme, per technology, etc. The Methodology will not prescribe 
one of those ways as necessary."

This is what I propose:

-------------------------------------

5.3 Procedure for evaluation

Once the scope of a conformance claim has been set and a sample of web 
pages or elements of web pages has been selected, the procedure for 
evaluation begins.

The procedure applies to the entire sample and checks each page and 
element against all applicable WCAG Success Criteria.


5.3.1 No precribed order of evaluation

The procedure does not require a particular order of evaluation. It may 
run through all pages testing one Success Criterion at a time, or tackle 
the sample page by page, working through all applicable Success Crtieria 
for one page at a time. The order in which pages and Success Criteria 
are covered is not prescribed. The only requirement is that all 
applicable Success Criteria are tested for all pages and elements of the 
sample.


5.3.2 Applicability of Success Criteria

For the testing of full pages in the sample, all Success Crtieria on the 
chosen level of conformance are applicable. This means that each full 
page in the sample is tested against all Success Criteria on the 
selected WCAG conformance level A, AA or AAA.

For the testing of selected elements (for example, a table or a form) 
the selected element must tested against all Success Criteria that are 
applicable to it.

In the case of a data table selected as element to be tested for 
conformance on level AA, this would mean that the success criteria 1.3.1 
"Info and relationships", 1.3.2 "Meaningful Sequence" and SC 1.4.3 
"Contrast (Minimum)" clearly apply, while Success Criteria 2.4.2 "Page 
Titled", 2.4.5 "Multiple Ways", 3.1.1 "Language of Page" clearly do not 
apply.

For a range of Success Criteria, applicability will depend on the 
content of the element selected. For example, for a table containing 
links, Success Criteria 2.1.1 "Keyboard", 2.4.3 "Focus Order", 2.4.4 
"Link Purpose (In Context)" and 2.4.7 "Focus Visible" would also apply.


5.3.3 Use of Sufficient Techniques and Failures

The evaluation of conformance of a particular page or element should 
draw on the list of Sufficient Techniques provided for each Success 
Criterion in the WCAG Quickref (http://www.w3.org/WAI/WCAG20/quickref/).

When the evaluation of the page or element unter test shows that one of 
the documented Sufficient Techniques has been used successfully, the 
Success Crtierion is met. Whereever possible, success should be 
determined by applying the tests provided at the end of Sufficient 
Techniques.


5.3.4 Establishing success or failure to of content under test

The success of content under test in implementing a Sufficient Technique 
(or set of Techniques that are deemed sufficient when used together) 
demonstrates the conformance of the page or element to the respective 
Success Criterion.

However, the failure of implementing a Sufficient Technique does not 
mean that the Success Crtierion is not met since other Techniques might 
have been used to achieve conformance, including Techniques not yet 
documented in the WCAG Quickref.

In addition to the Sufficient Techniques, the procedure must also check 
whether any of the documented WCAG Failures apply. If the test provided 
at the end of a WCAG Failure shows that the failure condition applies, 
then the page or element under test fails the associated Success Criterion.
Received on Thursday, 16 February 2012 09:21:19 GMT

This archive was generated by hypermail 2.3.1 : Friday, 8 March 2013 15:52:13 GMT