RE: Testing web page accessibility by phone - Usability

I think there are many threads here that are overlapping and may have little
to do with accessibility directly.  I had the opportunity to go the ACM
Special Interest Group for Computer Human Interaction conference in
Minneapolis in April, and most of the sessions there were pulling on more
general parts of this set of questions.  They were talking about usability,
of web pages, speech systems, wearable computers, of Open-source systems
like Linux and Mozilla,  and not in terms of the blind or deaf or
cognitively-limited, but in general terms, often with very intelligent,
non-disabled individuals like the researchers themselves being the users.
SIG CHI is largely an academic group, with corporate research groups,
universities and the like, and the proceedings are peer-reviewed.  But, on
the large topic of usability, all I could see was a lack of discipline.  It
is an art form, making gurus out of Ben Schneiderman and Jakob Neilsen,
because, in my opinion, the science is still being developed.  

The one Interactive Poster titled "Usability Inspections by Groups of
Specialists: Perceived Agreement in Spite of Disparate Observations" was
very pertinent.  In this study, Hertzum, Jacobsen and Molich (all from
Denmark) asked a group of usability specialists to evaluate a large web site
for ordering rental cars.  The eleven professional usability specialists all
looked at the same site, and were all given the same five user tasks to
perform.  But, they found totally separate usability problems.  Of the 220
unique problems, only a 9% overlap in problems found occurred between any
two evaluators.  However, in a group discussion on the problems, the
evaluators generally thought they were all in agreement about the problems
in the web site. 

To me this means that usability is still not a disciplined science and what
is easy for one to use may not be easy of another.  No wonder we are having
these discussions.

Dan Nissen

-----Original Message-----
From: David Poehlman [mailto:poehlman1@comcast.net]
Sent: Wednesday, May 29, 2002 7:52 AM
To: liz.fuller@labs.gte.com; w3c-wai-ig@w3.org
Subject: Re: Testing web page accessibility by phone



the kind of validation I am referring to here is inclusive and begins
with what is usually thought of as validation.  It goes like this.
Write your pages to speck, validate the pages to make sure they are
written to speck and while you are writing to speck, include in the
speck known factors that make pages usable/accessible and when you
validate the final product, use tools that will check for known
usability and accessibility factors and also that will allow you to
examine other issues that may be relevant.

People are important in the mix too but it all starts with a well laid
plan and when the users finally test the pages, they should find
insugnificant things that you may have missed or hadn't thought of but
that wound not necessarily break the accessibility/usability of the
pages.  I cannot talk on the phone and test a web page at the same time.

----- Original Message -----
From: "Liz Fuller" <liz.fuller@verizon.com>
To: <w3c-wai-ig@w3.org>
Sent: Wednesday, May 29, 2002 8:42 AM
Subject: RE: Testing web page accessibility by phone



Validation is all fine and good but it would be quite simple to have a
site
that validates and passes all the standards tests but when a user sits
in
front of it they can't use it at all.  Perhaps what Scott is testing in
his
phone sessions is not accessibility per se but usability by people who
are
blind.  This is an extremely valuable activity that will provide way
more
insight into how people will access your site than a simple validation
activity.  Both automated validation testing and real live user testing
are
important steps in good site design.

Liz


-----Original Message-----
From: w3c-wai-ig-request@w3.org [mailto:w3c-wai-ig-request@w3.org]On
Behalf
Of David Poehlman
Sent: Wednesday, May 29, 2002 8:03 AM
To: Jerry Weichbrodt; w3c-wai-ig@w3.org
Subject: Re: Testing web page accessibility by phone


interesting point Jerry!

This harkens back to my point of "vallidation" with which I was putting
forward the crazy notion that doing it right to begin with makes a huge
difference.

----- Original Message -----
From: "Jerry Weichbrodt" <gerald.g.weichbrodt@ived.gm.com>
To: <w3c-wai-ig@w3.org>
Sent: Wednesday, May 29, 2002 7:57 AM
Subject: Re: Testing web page accessibility by phone



I wonder how much of this depends on the user agent, by which I mean the
combination of the web browser and screen reader.  As a specific
example,
ever since JAWS for Windows discovered how to grab headings (*real*
headings
with numbers, not just over-sized type that web designers like to
pretend
are headings) and present them in a list with the ability to move
immediately to a heading, I have suddenly become very enthusiastic about
proper use of headings to mark off important section divisions in a web
page.  Prior to the heading support, I have to confess that headings
didn't
do much for me because they were essentially indistinguishable from
other
stuff on the page.  It makes a big difference, to me anyway, if you can
gain
some sort of hierarchical view of a web page rather than just the
classic
never-ending linear version.

In summary, how a web page "stacks up" may be surprisingly dependent on
the
browser/screen reader used to view it.  Just my two cents, and hopefully
not
too far off the mark.

Regards,
Jerry

----- Original Message -----
From: "phoenixl" <phoenixl@sonic.net>
To: <phoenixl@sonic.net>; <poehlman1@comcast.net>; <w3c-wai-ig@w3.org>
Sent: Tuesday, May 28, 2002 9:13 PM
Subject: Re: Testing web page accessibility by phone


> Hi,
>
> First, the goal isn't necessarily validation but looking at
> accessibility.  Second, different people have different
interpretations
> of what an accessible web page is to blind people.  For example, look
at
> the various standards that have been or are being developed.
>
> The methodology being used was to get subjective experience of various
> blind subjects.  Rather than taking the perspective that the web pages
> are accessible because they meet some set of standards, we focused on
> whether the blind subjects themselves experienced the web pages as
being
> accessible.  The feedback that was given was interesting and helpful.
>
> Using a comparison strategy can also be helpful, but not always
> necessary for getting useful information.  The issue of skill is
> something to consider.  A question though is how much is it a blind
> person's responsibility to have a certain skill level and how much is
it
> the web page's responsibility not to have high expectations for skill
> level?
>
> The Hisoftware person I talked with left me with the impression that
the
> software doesn't have mechanisms for measuring such things as how long
> does it take for the blind subject to understand a web page or
> determining how accurately the blind subject understands the web page.
> The software basically is checking syntax against a specified set of
> standards rather than evaluating the experience of the subjects.
>
> Rather than discussing the questions, it might be interesting first to
> use them when working with a variety of blind subjects.  However, a
> question to ask is if a sighted person can understand the purpose of a
> web page in let's say 15 seconds and it often takes a blind person 2
> minutes to understand the same web page, is that web page accessible?
>
> Scott

Received on Wednesday, 29 May 2002 09:28:33 UTC