W3C home > Mailing lists > Public > public-wai-evaltf@w3.org > September 2011

AW: Additional Point/Question: problem centered / page centered evaluation

From: Kerstin Probiesch <k.probiesch@googlemail.com>
Date: Sun, 11 Sep 2011 12:29:39 +0200
To: "'Vivienne CONWAY'" <v.conway@ecu.edu.au>, "'Michael S Elledge'" <elledge@msu.edu>, <public-wai-evaltf@w3.org>
Message-ID: <4e6c8d1f.835cdf0a.433b.633e@mx.google.com>
Hello TF,

when I'm checking a website (WCAG 2.0 conformance) the report is in most
cases not only checking for pass/fail along a checklist. I'm speaking with
the client to find out what might be the best: Just the checklist results or
the results incl. description of barriers and concrete measures to take for
developers and other groups involved with the website. This includes also
the formulation of preventive measures. If the client just want "a first
impression" also short-tests with e.g. 10 points are possible or limited by
hours. In these cases I give a clear statement that it is a just a short
accessibility review. Sometimes a short test is a good instrument to find
out what might be best for the future: a WCAG 2.0-Test, Consulting,

One work package of this TF could be: collecting different settings and
recommendations according to the settings, incl. re-tests. 



> -----Ursprüngliche Nachricht-----
> Von: public-wai-evaltf-request@w3.org [mailto:public-wai-evaltf-
> request@w3.org] Im Auftrag von Vivienne CONWAY
> Gesendet: Samstag, 10. September 2011 02:17
> An: Michael S Elledge; public-wai-evaltf@w3.org
> Betreff: RE: Additional Point/Question: problem centered / page
> centered evaluation
> Michael
> You bring up an excellent point - the post-assessment environment.  How
> do we assist clients to maintain the accessibility of their site, how
> often should a site be re-tested.  I think developing accessibility
> techniques such as checklists and important points for those adding
> content to existing sites, teaching them how to evaluate what they are
> adding is a good starting point.  IMHO this should be added to what
> we're doing as it's an important of establishing a methodology for
> testing websites - the on-going testing and maintenance.  Thoughts?
> Regards
> Vivienne L. Conway
> ________________________________________
> From: public-wai-evaltf-request@w3.org [public-wai-evaltf-
> request@w3.org] On Behalf Of Michael S Elledge [elledge@msu.edu]
> Sent: Saturday, 10 September 2011 12:35 AM
> To: public-wai-evaltf@w3.org
> Subject: Re: Additional Point/Question: problem centered / page
> centered   evaluation
> Hi All--
> Once again I concur with Denis' point. Clients aren't capable of paying
> for site-wide, manual evaluation, and evaluation tools are limited in
> what they verify. This means, of course, that clients ultimately have
> the responsibility for making their sites accessible. And that, as
> evaluators, we have a limited amount of leverage over the accessibility
> of a website.
> All is not lost, however. We have found that reviewing an accessibility
> report and repairing a site can give clients greater awareness and
> commitment to being compliant in the future.
> Which leads to another question/suggestion: shouldn't we also address
> the post-assessment environment? I.e., the need to embed commitment,
> technique and knowledge into organizational culture and processes?
> Mike
> On 9/9/2011 9:27 AM, Denis Boudreau wrote:
> > Hi Vivienne, all,
> >
> > On 2011-09-09, at 5:00 AM, Vivienne CONWAY wrote:
> >
> >> I think you explained that nicely.  I am always puzzling about how
> someone can say a website meets WCAG 2.0 AA if they haven't tested
> every single page for every single SC. Testing representative pages
> will give us a good 'idea' of the accessibility of the website, but I
> don't think anyone could/should certify a website using this method.
> What if we missed a page with a critical problem?  And again, how do
> you locate every problem/page?  Being able to locate issues is one of
> the best arguments for using automated tools to ASSIST in an evaluation
> - they tend to help us see patterns/trends.  Thoughts?
> > As an organization that does accessibility certification with various
> international and north american standards (WCAG 2.0, CLF 2.0, SGQRI
> 008 and Section508), we've had to struggle with that for quite a while.
> So I feel I should react to your question. ;p
> >
> > As you know, doing a complete assessment on a single web page takes
> time. For us, going through our checklist takes about 2 hours per page,
> AT testing included. We charge around 300,00$ CDN to do that. Now
> imagine that the client has a rather small website, with "only" 500
> pages or so. Doing as you suggest would mean a 150,000$ project,
> probably twice as much as what it cost them to get the website in the
> first place. Of course, going through so many pages would mean we can
> get the job done quicker on most pages as you only have to do your
> templates once, but still, with so many tests to run on content, you
> can hardly go below 1 hour per page... So, given we can cut down on
> time, we'd still talking about 150,00$ per page, for about 500 pages,
> so a whooping 75k. Totally insane.
> >
> > So while it's true that in order to be sure that you don't miss out
> on critical problems, you'd have to go through each and every one of
> them, it's not just realistic to think that can be done... nobody has
> that kind of money. And those that might have much larger websites,
> with tens of thousands of pages... "totally insane" takes on a totally
> new meaning.
> >
> > Which is why I was mentioning only getting a sample of critical
> pages, based on various templates and then, asking the client to make
> the required improvement on all pages themselves. Yes, we might be
> missing on a few things, but it still gives incredibly useful and
> relevant results that allow for a website to become much better
> accessibility wise. There are tools out there (Deque, HiSoftware and
> IBM being a few of the companies providing them) that can monitor
> thousands of pages in a very short time and those prove exceptionally
> useful to get a general idea and we're hoping to be able to add that to
> our methodology ourselves soon. If you can afford those tools, then you
> could do a thorough manual audit on sample pages while completing it
> with an automated sweep of the whole site. Then you actually reduce
> your error margin by a lot and your certification process would be more
> reliable.
> >
> > As an organization, we'Re not quite there yet, but we're working on
> this. When we label a website as WCAG 2.0 compliant for example, we'll
> note that the certification is based on a sample of pages, audited at a
> specific point in time. We can even provide the source code that was
> audited because of course, websites tend to degrade over time so unless
> the organization does a great job at maintaining, the website rarely
> remains compliant over time. And we deliver a label for a one year
> period, after which a new audit is required to renew the label.
> >
> > I agre, this is not the ideal solution, but I think it's the best
> that can be done at this moment. This is one of the reasons we're part
> of this group. We are very aware of the limitations of this method and
> look forward to improving it quite considerably over the next two years
> with you all.
> >
> > Questions and comments are welcomed.
> >
> >
> This e-mail is confidential. If you are not the intended recipient you
> must not disclose or use the information contained within. If you have
> received it in error please return it to the sender via reply e-mail
> and delete any record of it from your system. The information contained
> within is not the opinion of Edith Cowan University in general and the
> University accepts no liability for the accuracy of the information
> provided.
Received on Sunday, 11 September 2011 10:28:13 UTC

This archive was generated by hypermail 2.3.1 : Tuesday, 6 January 2015 20:40:18 UTC