W3C home > Mailing lists > Public > public-wai-evaltf@w3.org > September 2011

Re: Additional Point/Question: problem centered / page centered evaluation

From: Denis Boudreau <dboudreau@accessibiliteweb.com>
Date: Sat, 10 Sep 2011 16:24:43 -0400
Message-id: <D1A292FE-2207-4EC9-A98C-53CE24409D0F@accessibiliteweb.com>
To: Eval TF <public-wai-evaltf@w3.org>
Hi Mike, all,

The amount of leverage at our disposal is indeed rather limited as entirely depends on the good will of the website owner. However, experience shows that most clients who undertake an accessibility evaluation process and are willing to spend on something like this are usually very open to the idea of improving (and therefore, maintaining) the accessibility level of their website. What they nned is some understanding of what needs to be done and how to actually get it done. Some sort of a progressive and continued improvement roadmap (not sure his idea translates well in english, but for those of you that can understand french, I'm referring to a "plan d'amélioration progressif et continu". 

So really, yes, all hope is not lost here. By providing recommendations on things that need to be fixed (and maybe even helping them prioritizing those recommendations so the work gets done efficiently), we are actually handing out a roadmap to progressively improve things. And that is tremendously helpful, especially in cases where the organization has no clue where to begin.

So, in a way, I guess this may address the post-assessment environment you were referring to, by at least pointing towards a way of doing things...

Best regards,

/Denis


On 2011-09-09, at 12:35 PM, Michael S Elledge wrote:

> Hi All--
> 
> Once again I concur with Denis' point. Clients aren't capable of paying for site-wide, manual evaluation, and evaluation tools are limited in what they verify. This means, of course, that clients ultimately have the responsibility for making their sites accessible. And that, as evaluators, we have a limited amount of leverage over the accessibility of a website.
> 
> All is not lost, however. We have found that reviewing an accessibility report and repairing a site can give clients greater awareness and commitment to being compliant in the future.
> 
> Which leads to another question/suggestion: shouldn't we also address the post-assessment environment? I.e., the need to embed commitment, technique and knowledge into organizational culture and processes?
> 
> Mike
> 
> On 9/9/2011 9:27 AM, Denis Boudreau wrote:
>> Hi Vivienne, all,
>> 
>> On 2011-09-09, at 5:00 AM, Vivienne CONWAY wrote:
>> 
>>> I think you explained that nicely.  I am always puzzling about how someone can say a website meets WCAG 2.0 AA if they haven't tested every single page for every single SC. Testing representative pages will give us a good 'idea' of the accessibility of the website, but I don't think anyone could/should certify a website using this method.  What if we missed a page with a critical problem?  And again, how do you locate every problem/page?  Being able to locate issues is one of the best arguments for using automated tools to ASSIST in an evaluation - they tend to help us see patterns/trends.  Thoughts?
>> As an organization that does accessibility certification with various international and north american standards (WCAG 2.0, CLF 2.0, SGQRI 008 and Section508), we've had to struggle with that for quite a while. So I feel I should react to your question. ;p
>> 
>> As you know, doing a complete assessment on a single web page takes time. For us, going through our checklist takes about 2 hours per page, AT testing included. We charge around 300,00$ CDN to do that. Now imagine that the client has a rather small website, with "only" 500 pages or so. Doing as you suggest would mean a 150,000$ project, probably twice as much as what it cost them to get the website in the first place. Of course, going through so many pages would mean we can get the job done quicker on most pages as you only have to do your templates once, but still, with so many tests to run on content, you can hardly go below 1 hour per page... So, given we can cut down on time, we'd still talking about 150,00$ per page, for about 500 pages, so a whooping 75k. Totally insane.
>> 
>> So while it's true that in order to be sure that you don't miss out on critical problems, you'd have to go through each and every one of them, it's not just realistic to think that can be done... nobody has that kind of money. And those that might have much larger websites, with tens of thousands of pages... "totally insane" takes on a totally new meaning.
>> 
>> Which is why I was mentioning only getting a sample of critical pages, based on various templates and then, asking the client to make the required improvement on all pages themselves. Yes, we might be missing on a few things, but it still gives incredibly useful and relevant results that allow for a website to become much better accessibility wise. There are tools out there (Deque, HiSoftware and IBM being a few of the companies providing them) that can monitor thousands of pages in a very short time and those prove exceptionally useful to get a general idea and we're hoping to be able to add that to our methodology ourselves soon. If you can afford those tools, then you could do a thorough manual audit on sample pages while completing it with an automated sweep of the whole site. Then you actually reduce your error margin by a lot and your certification process would be more reliable.
>> 
>> As an organization, we'Re not quite there yet, but we're working on this. When we label a website as WCAG 2.0 compliant for example, we'll note that the certification is based on a sample of pages, audited at a specific point in time. We can even provide the source code that was audited because of course, websites tend to degrade over time so unless the organization does a great job at maintaining, the website rarely remains compliant over time. And we deliver a label for a one year period, after which a new audit is required to renew the label.
>> 
>> I agre, this is not the ideal solution, but I think it's the best that can be done at this moment. This is one of the reasons we're part of this group. We are very aware of the limitations of this method and look forward to improving it quite considerably over the next two years with you all.
>> 
>> Questions and comments are welcomed.
>> 
>> 
> 
Received on Saturday, 10 September 2011 20:25:17 GMT

This archive was generated by hypermail 2.3.1 : Friday, 8 March 2013 15:52:11 GMT