Re: Step 1.d. Define evaluation methods

Hi all,

In my view, this optional section (step 1.d) should focus on a scenario where the evaluation commissioner uses a custom technique to meet a success criterion that is not covered by documented WCAG Techniques. Only in this case could it be useful to describe the particular methods used in evaluation. Example: A custom control in an intranet application (say, a combobox) has been implemented and made accessible with WAI-ARIA so that it works well with a particular browser and screen reader deployed by the organisation. The evaluation technique may then describe a sample task and the expected output when using the defined UAs and ATs.

The way it stands, it is not clear to what length evaluators should go in describing their evaluation methods. If a particular testing organisation has opertationalized the testing procedure, would it be enough to reference this procedure generally, or should there be details on specific checkpoints, listing tools etc. I think this would be far too tedious and also often not really useful for users of the report (unless the test is aimed at safeguarding maximum repeatability so that other testers will arrive at the exact same results).

So a rewording suggestion would be:

----------

Step 1.d: Define Evaluation Methods to be Used (Optional)

Methodology Requirement 1.d: When web content uses custom techniques to meet particular WCAG 2.0 Success Criteria, any specific evaluation methods used to evaluate the conformance of these custom techniques should be described (Optional).

For custom techniques which are not part of the publicly documented (non-normative) Techniques for WCAG 2.0, it can be useful to document specific evaluation methods, for example, if these require the use of particular combinations of user agents and assisstive technolgies known to be used by the target audience of the content under test. Such evaluation methods might be made available by the site owner, their web developers, or the evaluation comissioner. Documenting specific evaluation methods to be used helps ensure consistent expectations between the evaluator and the evaluation commissioner.

This does not limit the evaluator from using other additional evaluation methods at a later point, for example, to evaluate particular content that was not identified at this early stage of the evaluation process.


----------

Best,
Detlev


--
Detlev Fischer
testkreis c/o feld.wald.wiese
Thedestr. 2, 22767 Hamburg

Mobil +49 (0)1577 170 73 84
Tel +49 (0)40 439 10 68-3
Fax +49 (0)40 439 10 68-5

http://www.testkreis.de
Beratung, Tests und Schulungen für barrierefreie Websites

Velleman, Eric schrieb am 10.04.2014 16:23:

> Your input for Step 1.d. Define evaluation methods to be used (Optional)
> 
> Relevant links:
> - <http://www.w3.org/TR/WCAG-EM/#step1d>
> - <http://www.w3.org/WAI/ER/conformance/comments-20140130#comment71>
> 
> Question: Do we have to explain this section more clearly? Also see tentative
> resolution in the DoC. Or is this section clear to you all?
> 
> 
> Eric
>

Received on Friday, 11 April 2014 07:47:10 UTC