RE: Draft for 5.3 Procedure for evaluation

Hi Detlev, Kerstin,

Good discussion, below some comments from me:

Hi Detlev, all,

just a few comments.

As long as we haven't decide upon the size of the sample it's a bit
difficult...If we have a sample of - let's say - 30 pages, why should it be
necessary to check every SC on every page? This would mean checking same
things 30 times. Much time for redundant work which a client has to pay.

EV: Agree, we agreed earlier that after a certain number of sampled pages, we would focus on elements and not on the complete page anymore. It is already adressed minimally in subclause 4.2 size of evaluation samples: " If pages are the same and do not show new errors, the evaluator can limit the evaluation of a page to the selection of particular elements or technologies that are targeted." 
I will propose a more detailed text for that in the next version of the document in subclause 4.2 and here in 5. Also the number of pages is dependant on the scope of the conformance claim and the availability of pages from the three different samples.

EV: I also agree with Detlev when he proposes not to set a specific order.

> 5.3.3 Use of Sufficient Techniques and Failures

EV: Do we just use Sufficient Techniques or also failures?

Does this mean that Sufficient Techniques will be Checkpoints?

In this case, as said before and written in the WCAG 2.0 and mentioned in
mails from the WG: Techniques are informative and not required for
conformance. Therefore Techniques can't be Checkpoints. Techniques as
Checkpoints would carry the message that they might be applicable, which
they are not.

Another point: if Techniques would be Checkpoints the testing protocol needs
to be reviewed every time when a another version of the techniques was
published which wouldn't be sustainable. Same for failures as CPs.

EV: This title is a proposal of what we have to discuss. How deep do we want to dive into the evaluation description. Do we include that evaluators have to use the techniques if available? and then report on the version, date etc. or is it completely free? 
This relates to the reporting: What is the level of detail in the report? Is it success criteria? or is it tests from the techniques? Or is that the choice of the site owner? 
This relates to the purpose of the report: Is it to be used for claiming conformance, repairing failures on a website or for benchmarking or.. ?

Best

--Kerstin

EV: Thanks Detlev and Kerstin



> This is what I propose:
>
> -------------------------------------
>
> 5.3 Procedure for evaluation
>
> Once the scope of a conformance claim has been set and a sample of web
> pages or elements of web pages has been selected, the procedure for
> evaluation begins.
>
> The procedure applies to the entire sample and checks each page and
> element against all applicable WCAG Success Criteria.


> 5.3.1 No precribed order of evaluation
>
> The procedure does not require a particular order of evaluation. It may
> run through all pages testing one Success Criterion at a time, or
> tackle
> the sample page by page, working through all applicable Success
> Crtieria
> for one page at a time. The order in which pages and Success Criteria
> are covered is not prescribed. The only requirement is that all
> applicable Success Criteria are tested for all pages and elements of
> the
> sample.

> 5.3.2 Applicability of Success Criteria
>
> For the testing of full pages in the sample, all Success Crtieria on
> the
> chosen level of conformance are applicable. This means that each full
> page in the sample is tested against all Success Criteria on the
> selected WCAG conformance level A, AA or AAA.

> For the testing of selected elements (for example, a table or a form)
> the selected element must tested against all Success Criteria that are
> applicable to it.
>
> In the case of a data table selected as element to be tested for
> conformance on level AA, this would mean that the success criteria
> 1.3.1
> "Info and relationships", 1.3.2 "Meaningful Sequence" and SC 1.4.3
> "Contrast (Minimum)" clearly apply, while Success Criteria 2.4.2 "Page
> Titled", 2.4.5 "Multiple Ways", 3.1.1 "Language of Page" clearly do not
> apply.
>
> For a range of Success Criteria, applicability will depend on the
> content of the element selected. For example, for a table containing
> links, Success Criteria 2.1.1 "Keyboard", 2.4.3 "Focus Order", 2.4.4
> "Link Purpose (In Context)" and 2.4.7 "Focus Visible" would also apply.
>
>
> 5.3.3 Use of Sufficient Techniques and Failures
>
> The evaluation of conformance of a particular page or element should
> draw on the list of Sufficient Techniques provided for each Success
> Criterion in the WCAG Quickref
> (http://www.w3.org/WAI/WCAG20/quickref/).
> When the evaluation of the page or element unter test shows that one of
> the documented Sufficient Techniques has been used successfully, the
> Success Crtierion is met. Whereever possible, success should be
> determined by applying the tests provided at the end of Sufficient
> Techniques.


> 5.3.4 Establishing success or failure to of content under test
>
> The success of content under test in implementing a Sufficient
> Technique
> (or set of Techniques that are deemed sufficient when used together)
> demonstrates the conformance of the page or element to the respective
> Success Criterion.
>
> However, the failure of implementing a Sufficient Technique does not
> mean that the Success Crtierion is not met since other Techniques might
> have been used to achieve conformance, including Techniques not yet
> documented in the WCAG Quickref.
>
> In addition to the Sufficient Techniques, the procedure must also check
> whether any of the documented WCAG Failures apply. If the test provided
> at the end of a WCAG Failure shows that the failure condition applies,
> then the page or element under test fails the associated Success
> Criterion.
>
>
>
>
>
>
>

Received on Thursday, 16 February 2012 13:53:18 UTC