Re: WCAG usability Re: Multiple versions of a web page

Hi,

If I recall correctly, some of us were talking about the idea of how
to measure the increase in accessibility of web pages at the WAI
kick-off gathering in Santa Clara.    It is a very thorney issue,
one very similar to measuring usability.

There were some interesting comments.  One was how often disabled
people are "measured" by non-disabled people and there was some resentment
about that.

I tried to suggest bringing in HCI people to help with doing
some research in understanding why there were problems using web pages.
There seemed to be a fair amount of resistance to bringing in
people from outside of the disabled community (unless they brought
money).

Another comment was basically along the lines that who should know better
than disabled web users what their problems are.  The basic assumption
seemmed to that testing of effectiveness wasn't needed for that reason.

Perhaps, it is time to do some soul searching and review what the
assumptions (and culture) the group has in its work.

Scott


> I don't think testing the usability is what I'm talking about.  I'm
> talking about testing 'what results come from this' not 'whether or not
> the guidelines themselves are usable.'  In other words, it's more of
> an ultimate effectiveness survey than a usability study.
> 
> I'm familiar with the issue on the WCAG issues list -- you may note
> that I'm credited (rightly or not -- I'm not sure it was a completely
> unique idea from me) with first raising the issue.
> 
> WCAG usability testing will focus on testing the ability of -web
> authors- to use -WCAG-; what I'm looking for is testable metrics,
> before and after, that measure the ability of -users- (with disabilities
> of various types) to access web sites which have -applied- WCAG.
> 
> To the best of my knowledge, there are no current plans for this nor
> any procedures for doing so, short of those defined in Mr. Nielsen's
> shorter paper on usability studies of people with disabilities.
> 
> In other words, the ability to say something like this next year:
> 
>       "Most web sites out there rate about 8.5 on the McCathieNevile-
>        Bartlett accessibility scale, meaning they're pretty difficult
>        to use if you have a disability.  Those designed according to
>        WCAG 2.0 specs are about 2 or 3, usually.  A 8.5 means
>        that on average it takes users with disabilities about 8 1/2
>        times longer to accomplish an action -- if they can accomplish
>        it at all -- than users without disabilities.  Note that
>        MCNB numbers are relative to the base usability of the site
>        itself, and thus a site with poor overall usability will be
>        that much more difficult for someone to use."
> 
> Or:
> 
>       "A study of 15 sites, before and after applying WCAG 2.0, showed
>        appreciable increase in the accessibility and usability of the
>        sites by people with disabilities.  Of the sites, 7 had already
>        complied with WCAG 1.0 Single-A, and these averaged an increase
>        of 64% more accessible, as measured by usability tests over a
>        range of 8 disability groups.  The 3 sites which had met
>        WCAG 1.0 Double-A and Triple-A compliance showed 25% increases.
>        Those 5 sites which had made no effort to meet accessibility
>        standards averaged an increase of 240% by applying WCAG 2.0.
>        Increases were measured through standard tests of content
>        accessibility and usability, and reflect increasing ease of
>        use and access to content by people with disabilities."
> 
> That's the kind of figures I want to hear.  The work proposed by David
> Sloan is also invaluable and definitely should be done, but it is a
> different kind of testing than what I propose here.
> 
> --Kynn

Received on Thursday, 27 December 2001 12:58:39 UTC