Re: Discussion about Additional Requirements for Evaluation (Optional)

Hi everyone,

I have thought about the implications of allowing WCAG-EM to be used for evluating parts of websites where the full page and site context may not (yet) be evailable to assess SCs that rely on that.

Here is what I have come up with, as draft input for the methodology should we decide this is a worthwhile addtion.
I have also tried to rephrase some sentences which state that WCAG-EM is a methodology for WCAG conformance evaluation  (see related discussion betweenShadi, Peter, Vivienne, Michael)

-------------
Introduction
-------------
This methodology, Website Accessibility Conformance Evaluation Methodology (WCAG-EM) 1.0, describes the minimum requirements of one possible approach to evaluate the conformance of existing websites to the Web Content Accessibility Guidelines (WCAG) 2.0. 

Change porposal: 
This methodology, Website Accessibility Conformance Evaluation Methodology (WCAG-EM) 1.0, describes the minimum requirements for the accessibility evaluation of websites (or of a defined scope of a website - compare Step 1.a: Define the Scope of the Website). The methodology applies the Success Criteria of the Web Content Accessibility Guidelines (WCAG) 2.0 and determines wether the web content evaluated meets these criteria. 
The evaluation typically covers only a representative sample of a much larger number of pages and page states that consitute the defined evaluation scope (compare Step 3: Select a Representative Sample). The selective sample means that WCAG EM cannot fully assert WCAG conformance for the defined scope (or only in cases where all pages and possible page states within the scope are included in the sample).

-------------
Introduction > Purpose of this document
-------------

WCAG-EM is for anyone who wants a common procedure for conformance evaluation of existing websites for WCAG 2.0

Change porposal: 
WCAG-EM is for anyone who wants a common procedure for the accessibility evaluation of websites according to the success criteria of WCAG 2.0.


Introduction > Purpose of this document > Third bullet point:
* Website developers who want to evaluate the accessibility conformance of their websites to monitor or improve them.

Change porposal: 
* Website developers who want to evaluate the accessibility conformance of their websites (or elements of websites) to monitor or improve them.  


-------------
Using This Methodology
-------------

This methodology is a thorough, robust approach for evaluating the conformance of websites to WCAG 2.0. 

Change porposal: 
This methodology is a thorough, robust approach for evaluating websites according to WCAG 2.0 success criteria. 

-------------
Scope of Applicability
-------------

The methodology defined by this document applies to full, self-enclosed websites. 

Change porposal: 
The methodology defined by this document typically applies to full, self-enclosed websites. In a context where WCAG-EM is used to check the accessibility of web content under development, the methodology may also be used with defined subset of success criteria (see section "Using WCAG-EM with a defined subset of success criteria")

Suggested new section at the end of 'Scope of Applicability':

Using WCAG-EM with a defined subset of success criteria

In development contexts where WCAG-EM is used to evaluate specific parts of a website, it can be sensible to excude particular WCAG success criteria because they do not apply to the part under evaluation. An example of a website part is a particular portlet that will be aggregated together with other parts in a customer-facing website. A design team may only have access to the website part under development and might therefore be unable to evaluate Success Criteria that require a consideration of the context of that part, i.e., the customer-facing page and often, other pages constituting the website (see step 1.e "Define WCAG Success Criteria to be excuded for the evaluation of websites during their development")

When limiting the scope of the evaluation by excluding WCAG Success Criteria that cannot be evaluated, the report should clearly state the limitation by enumerating all Success Criteria that were excluded. Also, the result of the evaluation cannot be used in any conformance claim. It is merely intended to provide targeted remedial information to the developers of the web site.

-------------
Step 1: Define the Evaluation Scope
-------------

If accepted, the option of using WCAG-EM with a defined subset of success criteria might be explained in more detail in a step 1.e at the end of "Step 1: Define the Evaluation Scope":

Step 1.e: Define WCAG Success Criteria to be excuded for the evaluation of websites during their development  (Optional)
If the evaluation is carried out to assess specific parts of a website during development, it may be impossible to evaluate Success Criteria that require a consideration of the eventual context of that part once it is aggregated together with other parts in a customer-facing website. Examples of success criteria that may be impossible to evaluate in a such design context are SC 2.4.2 Page Titled, SC 2.4.1 Bypass Blocks, SC 2.4.5 Multiple Ways, SC 2.4.8 Location, SC 3.1.1 Language of Page, SC 3.2.3 Consistent Navigation, and SC 3.2.4 Consistent Identification, which all require an assessment of web content in the context of the customer-facing page and often, in the context of other pages constituting the website.

What makes the inclusion a bit tricky is that some steps in Sampling in such as 3.c (Include Other Relevant Web Pages) and 3.e ( Include a randomly selected sample) may not be applicable in the website part evaluation context.

-------------
Step 4.a: Check for the Broadest Variety of Use Cases
-------------

For each web page in the sample selected per Step 3: Select a Representative Sample, check whether each of the WCAG 2.0 conformance requirements have been met, including all WCAG 2.0 Success Criteria relevant to the target conformance level (as per Step 1.c: Define the Conformance Target).

Change porposal: 
For each web page in the sample selected per Step 3: Select a Representative Sample, check whether each of the WCAG 2.0 conformance requirements have been met, including all WCAG 2.0 Success Criteria relevant to the target conformance level (as per Step 1.c: Define the Conformance Target) unless they have been explixcitly excluded in the evaluation of web site parrts in a development context (compare Step 1.e: Define WCAG Success Criteria to be excuded for the evaluation of websites during their development). 



--

Detlev Fischer
testkreis c/o feld.wald.wiese
Thedestr. 2, 22767 Hamburg

Mobil +49 (0)1577 170 73 84
Tel +49 (0)40 439 10 68-3
Fax +49 (0)40 439 10 68-5

http://www.testkreis.de
Beratung, Tests und Schulungen für barrierefreie Websites



----- Original Message -----
From: evelleman@bartimeus.nl
To: public-wai-evaltf@w3.org
Date: 09.09.2013 15:40:30
Subject: Discussion about Additional Requirements for Evaluation (Optional)


> For discussion: This section gives the possibility to define additional requirements for evaluation. In the Telco it was proposed that an evaluation commissioner may wants to limit the application requirements of the methodology? In fact that would not be a proper application of this methodology as described in "Application of this Methodology". Do we want to include a description of how to use this methodology in a non proper way?
> 
> This discussion addresses: http://www.w3.org/WAI/ER/conformance/ED-methodology-20130903#step1b

Received on Wednesday, 11 September 2013 11:13:51 UTC