W3C home > Mailing lists > Public > public-wai-evaltf@w3.org > October 2011

Alternative concise requirements

From: RichardWarren <richard.warren@userite.com>
Date: Tue, 4 Oct 2011 01:13:50 +0100
Message-ID: <DAA0A57D6439498FA29B29FC13D9EFBF@DaddyPC>
To: "Eval TF" <public-wai-evaltf@w3.org>
I am a little concerned that we are making life needlessly difficult by having produced eighteen requirements for a methodology to evaluate websites against twelve guidelines!  

Having read through the proposed requirements carefully I notice that some are very similar (such as supporting verification and supporting validity, or using unambiguous language and making it translatable).  Others are confusing, such as saying it will not stretch into techniques and tests (R01) and yet use existing testing techniques (R06). 

So I sat down and had a go at rationalising what we had produced and came up with ten requirements that cover (I hope) all the points raised in our various discussions. I am sure that it is not a perfect set of requirements, but I would like to share it with you just in case you think it might be the basis for a more concise list of requirements.


RQ 01 : Define methods for evaluating WCAG 2.0 conformance
The Methodology provides methods to measure conformance with WCAG 2.0. that can be used by the target audience (see section 2 above) for evaluating small or large websites, sections of websites or web-based applications.

RQ 02 – Unambiguous Interpretation
The methodology is written in clear language, understandable to the target audience and capable of translation to other languages.

RQ 03 – Reliable 
Different Web accessibility evaluators using the same methods on the same website(s) should get the same results. Evaluation process and results are documented to support independent verification.

RQ 04 - Tool and browser independent
The use and application of the Methodology is vendor-neutral and platform-independent. It is not restricted to solely manual or automated testing but allows for either or a combination of approaches.

RQ 05 -  QA framework specification guidelines
The Methodology will conform to the Quality Assurance framework specification guidelines as set in: http://www.w3.org/TR/qaframe-spec/.

RQ 06 - Machine-readable reporting
The Methodology includes recommendations for harmonized (machine-readable) reporting. It provides a format for delivering machine-readable reports using Evaluation and Report Language (EARL) in addition to using the standard template as at http://www.w3.org/WAI/eval/template.html

RQ 07 -  Use of existing WCAG 2.0 techniques
Wherever possible the Methodology will employ existing testing procedures in the WCAG 2.0 Techniques documents rather than replicate them.

RQ 08 -  Recommendations for scope and sampling
It includes recommendations for methods of sampling web pages in large websites and how to ensure that complete processes (such as for a shopping site where all the pages that are part of the steps in an ordering process) are included.  Such selections would be reflected in any conformance claim.

RQ 09 -  Includes tolerance metrics
It includes calculation methods for determining nearness of conformance.  Depending on the amount of tolerance, a failure could fall within a certain tolerance level meaning that the page or website might be considered conformant even though there is a failure. Such tolerances would be reflected in any conformance claim.

RQ 10 - Support documentation
The document will give a short description of the knowledge necessary for using the Methodology for evaluations.

Received on Tuesday, 4 October 2011 00:14:19 UTC

This archive was generated by hypermail 2.3.1 : Tuesday, 6 January 2015 20:40:18 UTC