W3C home > Mailing lists > Public > public-wai-ert@w3.org > March 2008

RE: WCAG 2 conformance and evaluation issues

From: Carlos Iglesias <carlos.iglesias@fundacionctic.org>
Date: Tue, 18 Mar 2008 18:49:07 +0100
Message-ID: <09700B613C4DD84FA9F2FEA5218828190320E11F@ayalga.fundacionctic.org>
To: "Shadi Abou-Zahra" <shadi@w3.org>
Cc: <public-wai-ert@w3.org>


Hi Shadi,

> > ...
> > - When we want to test or audit conformance, the current conformance
> claims form is not compatible with the processes that tools or auditors
> follow.
> >
> > Conformance is currently defined only for full Web pages and cannot be
> achieved if part of a Web page is excluded.
> >
> > If we look again at the WCAG 2 definition of web page, that includes for
> example any page that may be loaded into a frame (i.e. virtually anything)
> or any possible combination of interactions between the user and any sort
> of AJAX-like application, and thus is impossible to check in practice.
> >
> > Additionally, WCAG 2 conformance is focused on completeness in
> opposition of the sampling approach that is followed for manual inspection
> on real audits.
> 
> ...here is where I lost you.
> 
> If someone makes a claim that their Web page (be it a frameset, Web mail
> application, or shopping cart) conforms to WCAG 2.0, then they should be
> able to ensure that all its parts and functionality meets the acclaimed
> conformance level. This includes content that is aggregated, generated,
> or dynamically loaded. I hope we can agree on that.

Ideally yes. The problem is that while creating your own web site you normally combine several kinds of resources, typically:

- Static resources, a.k.a. templates
- CMS resources, code dynamically produced by your CMS
- Human edited resources, new content your editors introduce in a day by day basis.
- Aggregated contents from other sources (e.g. advertising providers, third part content...)
...

This take us to the situation where the only way a web site owner has to ensure that all these parts working together meet the acclaimed conformance level is to conduct a complete review that should be made from the user's perspective (i.e. you should test the front-end result).

This is mainly due to, even if you're able to the test all this resources individually (something that is not always possible for example with human edited resources), the combination of all them is not necessary accessible (something quite frequent).


> Now verifying that claim as an auditor is somewhat different. Usually it
> is not [economically] feasible to evaluate all possible instances and
> states of all Web pages in a given set (a.k.a. Web site). An evaluation
> methodology uses sampling techniques to create a kind of an (accurate or
> inaccurate) approximation of the actual conformance.

Similarly, it is not feasible to evaluate all possible instances and states of you own web site. The only difference with an external audit is that you have some extra information from the inside.


> So to me the question of sampling and auditing is part of an evaluation
> methodology rather than of WCAG 2.0 itself.

So this means that WCAG 2.0 conformance claims are focused on self-claiming, and not third part claims. Isn't it?


> In ERT WG we need to figure
> out how to record "what" was evaluated (a clear representation of a Web
> page, including it's current state) in order to support such evaluation
> methodologies. I do not understand your concern here...

It has no direct influence on EARL, but, correct me if I'm wrong, being accessibility testing related I thought it was on-topic for the group. 


Regards,
 CI.

__________________

Carlos Iglesias

Fundación CTIC
Parque Científico-Tecnológico de Gijón
33203 - Gijón, Asturias, España

teléfono: +34 984291212
fax: +34 984390612
email: carlos.iglesias@fundacionctic.org
URL: http://www.fundacionctic.org
Received on Tuesday, 18 March 2008 17:49:37 GMT

This archive was generated by hypermail 2.2.0+W3C-0.50 : Tuesday, 18 March 2008 17:49:37 GMT