W3C home > Mailing lists > Public > public-wai-ert@w3.org > March 2008

Re: WCAG 2 conformance and evaluation issues

From: Shadi Abou-Zahra <shadi@w3.org>
Date: Tue, 18 Mar 2008 17:20:19 +0100
Message-ID: <47DFEBC3.4040008@w3.org>
To: Carlos Iglesias <carlos.iglesias@fundacionctic.org>
CC: public-wai-ert@w3.org

Hi,

Carlos Iglesias wrote:
> 
> Hi group,
> 
> As per the last teleconference action item [1] here you have some thoughts about potential conformance and evaluation issues with WCAG 2.
> 
> The current WCAG 2 definition of web page is "A non-embedded resource obtained from a single URI using HTTP plus any other resources that are used in the rendering or intended to be rendered together with it by a user agent"
> 
> Including things like: 
> 
> - A frameset web page at "http://example.com/frameset" and all possible pages that may be rendered in any of the frames.
> 
> - A Web mail program built using Asynchronous JavaScript and XML (AJAX). The program lives entirely at "http://example.com/mail", but includes an inbox, a contacts area and a calendar. Links or buttons are provided that cause the inbox, contacts, or calendar to display, but do not change the URL of the page as a whole.
> 
> - A movie-like interactive shopping environment entirely at "http://shopping.example.com/" where you visually move about a store dragging products off of the shelves around you and into a visual shopping cart in front of you. Clicking on a product causes it to be demonstrated with a specification sheet floating alongside.
> 
>>From the accessibility testing perspective this arises two problems:
> 
> - When we want to record test results, if the test subject is a web page as defined by WCAG 2 it can't be represented by an URI solely any more because in a web applications developed with AJAX, FLEX or other technologies, the URI itself doesn't represent the current state of the application.
> 
> As we need to know what has exactly been tested, we may use a representation of the content, or at least a hash (or any other "status mark") to know if the status has changed.
> 
> To record the current state of an AJAX web page we can use all the correspondent HTTP requests and responses and the current DOM representation, but I'm not sure about whether something similar is available with other technologies like FLEX.

OK, so far I agree with your summary of the current situation. But...


> - When we want to test or audit conformance, the current conformance claims form is not compatible with the processes that tools or auditors follow.
> 
> Conformance is currently defined only for full Web pages and cannot be achieved if part of a Web page is excluded.
> 
> If we look again at the WCAG 2 definition of web page, that includes for example any page that may be loaded into a frame (i.e. virtually anything) or any possible combination of interactions between the user and any sort of AJAX-like application, and thus is impossible to check in practice.
> 
> Additionally, WCAG 2 conformance is focused on completeness in opposition of the sampling approach that is followed for manual inspection on real audits.

...here is where I lost you.

If someone makes a claim that their Web page (be it a frameset, Web mail 
application, or shopping cart) conforms to WCAG 2.0, then they should be 
able to ensure that all its parts and functionality meets the acclaimed 
conformance level. This includes content that is aggregated, generated, 
or dynamically loaded. I hope we can agree on that.

Now verifying that claim as an auditor is somewhat different. Usually it 
is not [economically] feasible to evaluate all possible instances and 
states of all Web pages in a given set (a.k.a. Web site). An evaluation 
methodology uses sampling techniques to create a kind of an (accurate or 
inaccurate) approximation of the actual conformance.

So to me the question of sampling and auditing is part of an evaluation 
methodology rather than of WCAG 2.0 itself. In ERT WG we need to figure 
out how to record "what" was evaluated (a clear representation of a Web 
page, including it's current state) in order to support such evaluation 
methodologies. I do not understand your concern here...


Best,
   Shadi


-- 
Shadi Abou-Zahra - http://www.w3.org/People/shadi/ |
   WAI International Program Office Activity Lead   |
  W3C Evaluation & Repair Tools Working Group Chair |
Received on Tuesday, 18 March 2008 16:20:51 GMT

This archive was generated by hypermail 2.2.0+W3C-0.50 : Tuesday, 18 March 2008 16:20:52 GMT