Re: Draft for 5.3 Procedure for evaluation

Hi All--

It seems to me that it is up to the evaluator (not the owner) to decide 
which techniques to use. Techniques are, after all, suggestions. Perhaps 
we can suggest, however, that evaluators consider using WCAG techniques 
and  provide an explanation when they are not.

Mike

On 2/16/2012 9:53 AM, Velleman, Eric wrote:
> Hi Detlev, Alistair, all,
>
> The question still remains if we want to say that it is necessary to test the techniques and if we want to describe how. This relates to the discussion about the scope where we decided that the website owner more of less decides what is being evaluated. Should the website owner also decide the level of reporting? Success criteria and/or techniques?
>
> Hartelijke groet,
>
> Eric
>
> ________________________________
> Van: Alistair Garrison [alistair.j.garrison@gmail.com]
> Verzonden: donderdag 16 februari 2012 15:28
> Aan: Detlev Fischer; Eval TF
> Onderwerp: Re: Draft for 5.3 Procedure for evaluation
>
> Hi Detlev,
>
> I broadly agree with all you have written below...
>
> With regard to "5.3.3 Use of Sufficient Techniques and Failures", I would like to correct something I previously wrote to you (and the list) regarding the usefulness, or not, of Failure conditions (http://lists.w3.org/Archives/Public/public-wai-evaltf/2012Jan/0057.html).
>
> I had, in fact, misunderstood their point. Re-reading the http://www.w3.org/TR/WCAG-TECHS/intro.html#intro_testing_techs document it clearly states: "By contrast, failure techniques do mean that a failure condition has occurred, in spite of any proper application of sufficient techniques."
>
> So, as you say, we must look to evaluate 'Sufficient Techniques and Failures' - as you can pass all the relevant sufficient techniques yet still fail WCAG if one or more failure conditions have been met.
>
> Hope this helps...
>
> Alistair
>
> On 16 Feb 2012, at 10:20, Detlev Fischer wrote:
>
> Dear EVAL TF,
>
> Here is an attempt to provide a draft for section 5.3 "Procedure for evaluation" of our methodology.
>
> The current version
> http://www.w3.org/WAI/ER/conformance/ED-methodology-20120209
> has the following draft text:
>
> "This subclause provides a step by step description of the evaluation of the website (sample). This does not include going into the guidelines, success criteria etc from WCAG 2.0 or related techniques. It could be possible to propose different ways to evaluate the guidelines: one by one, per theme, per technology, etc. The Methodology will not prescribe one of those ways as necessary."
>
> This is what I propose:
>
> -------------------------------------
>
> 5.3 Procedure for evaluation
>
> Once the scope of a conformance claim has been set and a sample of web pages or elements of web pages has been selected, the procedure for evaluation begins.
>
> The procedure applies to the entire sample and checks each page and element against all applicable WCAG Success Criteria.
>
>
> 5.3.1 No precribed order of evaluation
>
> The procedure does not require a particular order of evaluation. It may run through all pages testing one Success Criterion at a time, or tackle the sample page by page, working through all applicable Success Crtieria for one page at a time. The order in which pages and Success Criteria are covered is not prescribed. The only requirement is that all applicable Success Criteria are tested for all pages and elements of the sample.
>
>
> 5.3.2 Applicability of Success Criteria
>
> For the testing of full pages in the sample, all Success Crtieria on the chosen level of conformance are applicable. This means that each full page in the sample is tested against all Success Criteria on the selected WCAG conformance level A, AA or AAA.
>
> For the testing of selected elements (for example, a table or a form) the selected element must tested against all Success Criteria that are applicable to it.
>
> In the case of a data table selected as element to be tested for conformance on level AA, this would mean that the success criteria 1.3.1 "Info and relationships", 1.3.2 "Meaningful Sequence" and SC 1.4.3 "Contrast (Minimum)" clearly apply, while Success Criteria 2.4.2 "Page Titled", 2.4.5 "Multiple Ways", 3.1.1 "Language of Page" clearly do not apply.
>
> For a range of Success Criteria, applicability will depend on the content of the element selected. For example, for a table containing links, Success Criteria 2.1.1 "Keyboard", 2.4.3 "Focus Order", 2.4.4 "Link Purpose (In Context)" and 2.4.7 "Focus Visible" would also apply.
>
>
> 5.3.3 Use of Sufficient Techniques and Failures
>
> The evaluation of conformance of a particular page or element should draw on the list of Sufficient Techniques provided for each Success Criterion in the WCAG Quickref (http://www.w3.org/WAI/WCAG20/quickref/).
>
> When the evaluation of the page or element unter test shows that one of the documented Sufficient Techniques has been used successfully, the Success Crtierion is met. Whereever possible, success should be determined by applying the tests provided at the end of Sufficient Techniques.
>
>
> 5.3.4 Establishing success or failure to of content under test
>
> The success of content under test in implementing a Sufficient Technique (or set of Techniques that are deemed sufficient when used together) demonstrates the conformance of the page or element to the respective Success Criterion.
>
> However, the failure of implementing a Sufficient Technique does not mean that the Success Crtierion is not met since other Techniques might have been used to achieve conformance, including Techniques not yet documented in the WCAG Quickref.
>
> In addition to the Sufficient Techniques, the procedure must also check whether any of the documented WCAG Failures apply. If the test provided at the end of a WCAG Failure shows that the failure condition applies, then the page or element under test fails the associated Success Criterion.
>
>
>
>
>
>
>
>
>
>
>
>

Received on Thursday, 16 February 2012 15:03:09 UTC