Re: WCAG usability Re: Multiple versions of a web page

I don't think testing the usability is what I'm talking about.  I'm
talking about testing 'what results come from this' not 'whether or not
the guidelines themselves are usable.'  In other words, it's more of
an ultimate effectiveness survey than a usability study.

I'm familiar with the issue on the WCAG issues list -- you may note
that I'm credited (rightly or not -- I'm not sure it was a completely
unique idea from me) with first raising the issue.

WCAG usability testing will focus on testing the ability of -web
authors- to use -WCAG-; what I'm looking for is testable metrics,
before and after, that measure the ability of -users- (with disabilities
of various types) to access web sites which have -applied- WCAG.

To the best of my knowledge, there are no current plans for this nor
any procedures for doing so, short of those defined in Mr. Nielsen's
shorter paper on usability studies of people with disabilities.

In other words, the ability to say something like this next year:

      "Most web sites out there rate about 8.5 on the McCathieNevile-
       Bartlett accessibility scale, meaning they're pretty difficult
       to use if you have a disability.  Those designed according to
       WCAG 2.0 specs are about 2 or 3, usually.  A 8.5 means
       that on average it takes users with disabilities about 8 1/2
       times longer to accomplish an action -- if they can accomplish
       it at all -- than users without disabilities.  Note that
       MCNB numbers are relative to the base usability of the site
       itself, and thus a site with poor overall usability will be
       that much more difficult for someone to use."

Or:

      "A study of 15 sites, before and after applying WCAG 2.0, showed
       appreciable increase in the accessibility and usability of the
       sites by people with disabilities.  Of the sites, 7 had already
       complied with WCAG 1.0 Single-A, and these averaged an increase
       of 64% more accessible, as measured by usability tests over a
       range of 8 disability groups.  The 3 sites which had met
       WCAG 1.0 Double-A and Triple-A compliance showed 25% increases.
       Those 5 sites which had made no effort to meet accessibility
       standards averaged an increase of 240% by applying WCAG 2.0.
       Increases were measured through standard tests of content
       accessibility and usability, and reflect increasing ease of
       use and access to content by people with disabilities."

That's the kind of figures I want to hear.  The work proposed by David
Sloan is also invaluable and definitely should be done, but it is a
different kind of testing than what I propose here.

--Kynn

At 11:26 PM -0500 12/26/01, Charles McCathieNevile wrote:
>Kynn, you will then be as pleased as I am that the WCAG group is working with
>a few organisations (notably in the UK) to empirically test the usability of
>WCAG 2 in particular, following some research done and published on WCAG 1.0.
>
>It is listed as an issue by the group - you should be able to follow it
>there. http://www.w3.org/WAI/GL/wcag20-issues.html#14
>
>cheers
>
>Charles McCN
>
>On Mon, 24 Dec 2001, Kynn Bartlett wrote:
>
>   I think this is an excellent question that needs to be addressed by
>   the WCAG working group; I think it's vitally important that we have
>   some testable metrics not only to whether or not our guidelines are
>   being met, but also to what degree that improves (or doesn't improve)
>   access by our core audiences.
>
>   If Nielsen's work teaches us nothing else, it should teach us that
>   standard empirical methods of research and study can be applied to
>   accessibility and it's not just pie-in-the-sky hopes and personal
>   anecdotes. [*]
>
>   --Kynn


-- 
Kynn Bartlett <kynn@idyllmtn.com>                 http://kynn.com
Chief Technologist, Idyll Mountain            http://idyllmtn.com
Web Accessibility Expert-for-hire          http://kynn.com/resume
January Web Accessibility eCourse           http://kynn.com/+d201

Received on Thursday, 27 December 2001 00:25:18 UTC