- From: Kynn Bartlett <kynn-edapta@idyllmtn.com>
- Date: Wed, 26 Dec 2001 21:02:54 -0800
- To: Charles McCathieNevile <charles@w3.org>
- Cc: Scott Luebking <phoenixl@sonic.net>, <poehlman1@home.com>, WAI IG <w3c-wai-ig@w3.org>
I don't think testing the usability is what I'm talking about. I'm talking about testing 'what results come from this' not 'whether or not the guidelines themselves are usable.' In other words, it's more of an ultimate effectiveness survey than a usability study. I'm familiar with the issue on the WCAG issues list -- you may note that I'm credited (rightly or not -- I'm not sure it was a completely unique idea from me) with first raising the issue. WCAG usability testing will focus on testing the ability of -web authors- to use -WCAG-; what I'm looking for is testable metrics, before and after, that measure the ability of -users- (with disabilities of various types) to access web sites which have -applied- WCAG. To the best of my knowledge, there are no current plans for this nor any procedures for doing so, short of those defined in Mr. Nielsen's shorter paper on usability studies of people with disabilities. In other words, the ability to say something like this next year: "Most web sites out there rate about 8.5 on the McCathieNevile- Bartlett accessibility scale, meaning they're pretty difficult to use if you have a disability. Those designed according to WCAG 2.0 specs are about 2 or 3, usually. A 8.5 means that on average it takes users with disabilities about 8 1/2 times longer to accomplish an action -- if they can accomplish it at all -- than users without disabilities. Note that MCNB numbers are relative to the base usability of the site itself, and thus a site with poor overall usability will be that much more difficult for someone to use." Or: "A study of 15 sites, before and after applying WCAG 2.0, showed appreciable increase in the accessibility and usability of the sites by people with disabilities. Of the sites, 7 had already complied with WCAG 1.0 Single-A, and these averaged an increase of 64% more accessible, as measured by usability tests over a range of 8 disability groups. The 3 sites which had met WCAG 1.0 Double-A and Triple-A compliance showed 25% increases. Those 5 sites which had made no effort to meet accessibility standards averaged an increase of 240% by applying WCAG 2.0. Increases were measured through standard tests of content accessibility and usability, and reflect increasing ease of use and access to content by people with disabilities." That's the kind of figures I want to hear. The work proposed by David Sloan is also invaluable and definitely should be done, but it is a different kind of testing than what I propose here. --Kynn At 11:26 PM -0500 12/26/01, Charles McCathieNevile wrote: >Kynn, you will then be as pleased as I am that the WCAG group is working with >a few organisations (notably in the UK) to empirically test the usability of >WCAG 2 in particular, following some research done and published on WCAG 1.0. > >It is listed as an issue by the group - you should be able to follow it >there. http://www.w3.org/WAI/GL/wcag20-issues.html#14 > >cheers > >Charles McCN > >On Mon, 24 Dec 2001, Kynn Bartlett wrote: > > I think this is an excellent question that needs to be addressed by > the WCAG working group; I think it's vitally important that we have > some testable metrics not only to whether or not our guidelines are > being met, but also to what degree that improves (or doesn't improve) > access by our core audiences. > > If Nielsen's work teaches us nothing else, it should teach us that > standard empirical methods of research and study can be applied to > accessibility and it's not just pie-in-the-sky hopes and personal > anecdotes. [*] > > --Kynn -- Kynn Bartlett <kynn@idyllmtn.com> http://kynn.com Chief Technologist, Idyll Mountain http://idyllmtn.com Web Accessibility Expert-for-hire http://kynn.com/resume January Web Accessibility eCourse http://kynn.com/+d201
Received on Thursday, 27 December 2001 00:25:18 UTC