Re: WCAG 2 conformance and evaluation issues

Hi Carlos,

Carlos Iglesias wrote:
>> If someone makes a claim that their Web page (be it a frameset, Web
>> mail application, or shopping cart) conforms to WCAG 2.0, then they
>> should be able to ensure that all its parts and functionality meets
>> the acclaimed conformance level. This includes content that is
>> aggregated, generated, or dynamically loaded. I hope we can agree
>> on that.
> 
> Ideally yes. The problem is that while creating your own web site you
> normally combine several kinds of resources, typically:
> 
> - Static resources, a.k.a. templates
> - CMS resources, code dynamically produced by your CMS
> - Human edited resources, new content your editors introduce in a day 
> by day basis. 
> - Aggregated contents from other sources (e.g. advertising providers, 
> third part content...) ...
>
> This take us to the situation where the only way a web site owner has
> to ensure that all these parts working together meet the acclaimed
> conformance level is to conduct a complete review that should be made
> from the user's perspective (i.e. you should test the front-end
> result).
> 
> This is mainly due to, even if you're able to the test all this
> resources individually (something that is not always possible for
> example with human edited resources), the combination of all them is
> not necessary accessible (something quite frequent).

It isn't always an easy task to ensure accessibility, especially in Web
applications, user generated content, and aggregated content. How would 
changing the definition for Web page address this complexity? Do you 
have a specific suggestion to help move this discussion forward?


>> Now verifying that claim as an auditor is somewhat different.
>> Usually it is not [economically] feasible to evaluate all possible
>> instances and states of all Web pages in a given set (a.k.a. Web
>> site). An evaluation methodology uses sampling techniques to create
>> a kind of an (accurate or inaccurate) approximation of the actual
>> conformance.
> 
> Similarly, it is not feasible to evaluate all possible instances and
> states of you own web site. The only difference with an external
> audit is that you have some extra information from the inside.

I think this is a significant difference. While developing Web content,
ideally one uses an iterative combination of designing and evaluating
the individual parts as well as the integrated system as a whole. One
knows more about the internal mechanisms and can therefore make very
different assumptions on the overall functionality/accessibility.


>> So to me the question of sampling and auditing is part of an
>> evaluation methodology rather than of WCAG 2.0 itself.
> 
> So this means that WCAG 2.0 conformance claims are focused on
> self-claiming, and not third part claims. Isn't it?

That is an orthogonal discussion, I don't understand why you raise it. 
Yes, it is often easier for the developers to make statements about the 
functionality/accessibility of their own Web content than for external 
auditors. But WCAG 2.0 itself does not suggest preferences for one or 
the other mechanism. It simply says that if someone claims that some 
representation of information (a.k.a. "Web page") meets WCAG 2.0, then 
this claim applies to all of its sub-parts.


>> In ERT WG we need to figure out how to record "what" was evaluated
>> (a clear representation of a Web page, including it's current
>> state) in order to support such evaluation methodologies. I do not
>> understand your concern here...
> 
> It has no direct influence on EARL, but, correct me if I'm wrong,
> being accessibility testing related I thought it was on-topic for the
> group.

Yes, indeed. And this is why I said we need to make sure we can support 
evaluation methodologies. I just don't understand why the definition of 
Web page in WCAG 2.0 is an issue for you. What am I missing?


Regards,
   Shadi


-- 
Shadi Abou-Zahra - http://www.w3.org/People/shadi/ |
   WAI International Program Office Activity Lead   |
  W3C Evaluation & Repair Tools Working Group Chair |

Received on Tuesday, 18 March 2008 18:43:19 UTC