Proposed text on "WCAG-EM Conformance" [was Re: EvalTF Telco]

Hi Eric, all,

At our last meeting we discussed the fact that WCAG 2.0 defines a 
conformance claim "only for Web pages", and that it can only be made if 
a page fully satisfies /all/ of the success criteria (A/AA/AAA as 
appropriate to the claim).  We also observed that since WCAG-EM defines 
a sampling methodology, the output of a WCAG-EM evaluation couldn't be a 
"WCAG 2.0 conformance claim" for the website being evaluated, since by 
the nature of sampling not all pages had been evaluated.  And finally, 
we agreed that a new section describing this situation and what 
conformance means for WCAG-EM should be added.

I took the action from that meeting to draft proposed language for that 
section, which I include below.  Perhaps if there is time we might start 
discussing this tomorrow at our meeting.  Looking at our current editors 
draft <http://www.w3.org/WAI/ER/conformance/ED-methodology-20130712>, I 
suggest this new section should go into the Introduction section 
<http://www.w3.org/WAI/ER/conformance/ED-methodology-20130712#introduction>, 
appearing right after Terms and Definitions 
<http://www.w3.org/WAI/ER/conformance/ED-methodology-20130712#terms>.

================================


    "Conformance" in the context of WCAG-EM

WCAG 2.0 <http://www.w3.org/TR/WCAG> defines the term "conformance 
<http://www.w3.org/TR/WCAG/#conformancedef>", and also describes the 
requirements for a Web page to conform to WCAG 2.0 
<http://www.w3.org/TR/WCAG/#conformance-reqs>.  Furthermore, WCAG 2.0 
describes the requirements of an optional WCAG 2.0 conformance claim 
<http://www.w3.org/TR/WCAG/#conformance-claims>, which "is defined only 
for Web pages".  Under these terms and descriptions, a claim may only be 
made for specific pages which are explicitly known to satisfy /all of 
the requirements/ (at the A, AA, or AAA level as appropriate).  This 
means a "WCAG 2.0 conformance claim" cannot be made for a collection of 
web pages unless it is known that /all/ of those web pages satisfy /all 
/of the requirements.

The purpose of WCAG-EM is to define a methodology for sampling and 
evaluating large or complex websites and web applications using the WCAG 
2.0 success criteria (at the A, AA, and AAA levels as appropriate), 
based on a sample of that website / web application, /without 
/evaluating /all /of the web pages or possible user interactions with a 
web application.  This is often necessary because it is rarely if ever 
possible to exhaustively evaluate every page for every success criteria 
in large or complex websites, or to evaluate every possible user input 
and dynamic output of large or complex web applications.

As a result of this, WCAG-EM cannot be used to make a "WCAG 2.0 
conformance claim" for a website or web application, as only a subset of 
that website or web application has been evaluated.  When used 
successfully, however, WCAG-EM can ascertain the /extent to which a 
website or web application /*/likely /*/conforms/. This result of 
WCAG-EM can then be used to generate a "WCAG 2.0 conformance report" 
(see Appendix C: Example Reports 
<http://www.w3.org/WAI/ER/conformance/ED-methodology-20130712#reports> 
for the kinds of reports that can be generated).  The more detailed 
WCAG-EM reports described herein note the issues found in the 
evaluation, and provide some confidence level as to how representative 
these findings are for the entire website or web application under 
evaluation.

================================


Regards,

Peter

On 7/23/2013 2:14 PM, Velleman, Eric wrote:
> Dear Eval TF,
>
> The next teleconference is scheduled for Thursday 25 July 2013 at:
>    * 14:00 to 15:00 UTC
>    * 15:00 to 16:00 UK Time
>    * 16:00 to 17:00 Central European Time (time we use as reference)
>    * 10:00 to 11:00 North American Eastern Time (ET)
>    * 07:00 to 08:00 North American Pacific Time (PT)
>    * 22:00 to 23:00 Western Australia Time
>    * 00:00 to 01:00 Eastern Australia Time (Note: Friday)
>
> Please check the World Clock Meeting Planner to find out the precise date for your own time zone:
>    - <http://www.timeanddate.com/worldclock/meeting.html>
>
> The teleconference information is: (Passcode 3825 - "EVAL")
>    * +1.617.761.6200
>    * SIP / VoIP -http://www.w3.org/2006/tools/wiki/Zakim-SIP
>
> We also use IRC to support the meeting: (http://irc.w3.org)
>    * IRC server: irc.w3.org
>    * port: 6665
>    * channel: #eval
>
>
> AGENDA:
>
> #1. Welcome
>
> #2. Status of Survey 10
> Survey: https://www.w3.org/2002/09/wbs/48225/evaltfq10/
> What is accepted and what do we need to discuss further. Topics will be sent just before the telco and discussed in next agenda item.
>
> #3. Discuss topics from the survey and decide if they need to be further discussed on the list
>
> #4. Parameters of the sample size
> See discussion at: http://lists.w3.org/Archives/Public/public-wai-evaltf/2013Jul/0028.html
>
> #5. Other issues
>
> Kindest regards,
> Eric
>

-- 
Oracle <http://www.oracle.com>
Peter Korn | Accessibility Principal
Phone: +1 650 5069522 <tel:+1%20650%205069522>
500 Oracle Parkway | Redwood City, CA 94064
Green Oracle <http://www.oracle.com/commitment> Oracle is committed to 
developing practices and products that help protect the environment