W3C home > Mailing lists > Public > wai-eo-editors@w3.org > October 2001

Re: Call for review: Evaluating Web Sites for Accessibility

From: Liz Roberts <liz@netlogix.net>
Date: Tue, 30 Oct 2001 13:48:09 -0500
To: <wai-eo-editors@w3.org>
Message-ID: <B8046019.19BC4%liz@netlogix.net>
I first go through line by line with comments/questions and then make
general comments at the end.  I hope these are of help.

==============================

Question 2 of Preliminary Review:

---- 

> 3.  change the font size (larger and smaller) in the browser, and observe
whether the page is still readable.

My Comments:

This is confusing to me: of course you can keep shrinking the size of the
type and make it unreadable.  I'm confused as to what this supposed to
accomplish.

----

> 4.  set screen resolution to 640 x 480 and observe whether or not this forces
the page into horizontal scrolling

My Comments:

What a user sees at 640 x 480 varies greatly by browser, platform, and user
settings. How should they be taken into account?

This is the only one of the points without a period at the end of the
sentence.  I would say add a period, but you start each rule lower case, so
do you instead need to use to commas?

----

> 5.  change the display color to black and white (or print out page on black
and white printer) and observe whether color contrast is adequate.

My Comments:

Printing is useless to me: most printers I've come across print white text
on a dark background as black text on white (sometimes even when my
print-specific css specifies otherwise!).  Unscientific as it may be, I use
the "squinting test" that artists use to evaluate contrast.  You don't have
to be able to read the text, but can you identify where it is?  I don't know
if that's the thing to recommend, but I adamantly believe printing would
rarely be useful.

----

> 6.  put away the mouse and tab through the links and form controls on a page,
making sure that you can access all links and form controls, and that the
links clearly indicate what they lead to.

My Comments:

Semantics, but I'd say "without using the mouse."

----

I also want to add that I'm not sure who your target audience is here; you
mention they don't need to know coding, but they certainly need to have a
familiarity with screen resolution, "tabbing," equivalents (it's linked in
the next section, but not here; I think it should be linked), and the WCAG
(if you want users to familiarize themselves with the WCAG before doing the
preliminary review, say so).

Directing users to the Web Content Accessibility Guidelines at the beginning
of the preliminary review (not just mentioning it in the intro) seems to be
logical to me.  If you're not familiar with the color contrast issue, you
could easily print out your page and not see any problems with legibility.

I would like to see a mention of *what* reviewers should be looking for at
each step.

==============================

Question 3 of Preliminary Review:

----

> 3. Use a voice browser (such as Home Page Reader) or a text browser (such as
Lynx) and examine the Web site while answering these questions (NOTE:
experienced users of screen readers may substitute a screen reader for a
voice or text browser, but if blind, may need a sighted partner to compare
information available visually; if sighted, listen to it with eyes closed,
then open eyes and confirm whether the information is equivalent)...
>   1. is equivalent information available through the voice or text browser as
is available through the GUI browser?
>   2. is the information presented in a similar logical order as when viewed
through the GUI browser?

My Comments:

When I look at my carefully marked up *data* tables in Lynx (Mac), they are
confusing.  Visually, they're still in a semi-logical state, but they're
incredibly hard to read and don't appear to have logical connections... I'm
at a loss right now trying to figure out if they have requisite information.

I'm fairly confident my data tables are accessible (note: only *fairly*
confident), but *only* because I've tried so hard to research and code in an
accessible and valid way.  If the reviewer didn't know what to look for in
the code, I would have to say it would look quite poorly formatted in lynx.
(It looks quite poorly formatted to me and certainly raises lots of doubts!)

===============================

Comprehensive Evaluation Intro
----
> A comprehensive evaluation combines semi-automatic, manual, and usability
testing. Comprehensive evaluations require familiarity with Web mark-up
languages; initial downloading and/or training on a variety of evaluation
tools and approaches; configuration of browser settings; and coordination
with reviewers with a variety of disabilities. Evaluation with users is
important as it helps to identify problems in how the technical solutions
are being applied.

My Comments:

While undeniably important, it ironically creates its own "have and have
not" issue: usability testing from reviewers with a variety of disabilities.

This is exactly the reason I turn to the W3C & WAI for information: because
our projects rarely have the time or budget for traditional focus groups let
alone ones this extensive.  Requesting volunteers--as I've often heard as a
response--*every month* would shortly be met with little reaction (and
rightly so!).  Undoubtedly, this should be recommended, but don't forget
about the "little guys"; we certainly try not to!

Also, I don't think you need to use semicolons, but rather commas to
separate the items in your list.
===============================

Question 1 of the Comprehensive Evaluation
----
> 1. Identify scope of site to be evaluated, and the targeted conformance level
for the evaluation:
>   1.  identify a page selection which includes at least one of each different
type of page on the site, and all top pages or entry pages to the site.
>   2.  identify and clearly disclose either:
>     *  the entire Web site including all pages at a base URL;
>     *  or an expanded page selection, to be clearly explained and disclosed on
the Web site. Suggestions for inclusions in this expanded page selection:
pages from different sections of the Web site; pages representing different
"look & feel"; pages representing different development tools and processes
including those generated from databases; pages produced under different
guidelines; "contact us" pages; pages critical to your business; etc. If any
area of a site is excluded from evaluation, be sure to disclose this
information.

My Comments:

I'm a little lost here.

What is an "entry page"? Aren't they all?  Perhaps "otherwise excluded key
entrance or function pages." Yes, it's vague, but I would propose less-so.

At first blush, I thought we were determining a single finite area of the
site to be evaluated and was immensely confused about what each of these
first two point had to do with one another.  Later, when you discuss
evaluating these selections, I can't for the life of me figure out why you
insist on two selections.  Either you do the whole site or the expanded page
selection; what is the point of the smaller "page selection" scope?

===============================

Question 2 of the Comprehensive Evaluation
----
> 2. Semi-automatic and automatic evaluation
>   1. Validate markup including syntax and style sheets, using all applicable
validators, on page selection. Run at least one validation tool across
entire Web site or expanded page selection.
>     * HTML Validation service
>     * HTML Tidy 
>     * CSS Validation service
>     * MathML Validator
>   2.    Use at least two accessibility evaluation tools, on page selection,
and run at least one tool across entire Web site .
>     * select from examples listed under #4 in preliminary review above

My Comments:

AFAIK, none of these HTML or CSS validators spider (or evaluate more than
one document at once), so why, again, is there the need for two different
areas of the site--which overlap--to be evaluated more or less separately
from each other?  (As far as the accessibility evaluation tool, I do realize
that the downloadable Bobby, for instance, will run through a whole site.)

I don't think you need the commas around "on page selection" in point 2.

===============================

Question 5 of the Comprehensive Evaluation
----
> Summarize and follow-up
>   1. Summarize any problems and best practices identified for each page type
and a representative URL, and method by which they were identified
>   2. Recommend follow-up steps, potentially including:
>     * repair of accessibility barriers identified through conformance
evaluation process. NOTE: for evaluations using "expanded page selection"
instead of "entire Web site," apply what you've learned to other pages
>     * expanding best practices on site
>     * ongoing maintenance and monitoring of site

My Comments:

What exactly are the "page type[s]"?  Are you referring to the "page
selection" definition, "expanded page selection" definition, or neither?

===============================
Part three of Considerations for Specific Contexts
---
> Evaluation of legacy sites

> Occasionally Web sites that are "frozen" (legacy; no longer actively
maintained) are found to have substantial accessibility problems. It can be
difficult to determine how to address these. It is helpful to:

>  * identify who the current owner is, and whether they have any obligation or
interest in making the site accessible;
>  * after evaluating the site, outline the changes that would be required to
retrofit the site for accessibility;
>  * identify and propose resources and a timeline for retrofitting the site;
>  * disclose accessibility problems on the site.

My Comments:

How are points two (evaluate, outline) and four (diclose) different?  If
they are different, it seems that disclosing the accessibility problems
should come before identifying the resources and timeline to retrofit the
site.

===============================

I like the idea of concrete steps for companies to follow in order to gain
compliance with WCAG.  I'm concerned that even the preliminary review
requires a familiarity with the checkpoints that defeats the purpose of the
preliminary review questions.

I'm also concerned that there are lots of other questions raised by this
process, but how to find answers to those is not addressed.  The questions
raised I see spanning from general to specific: deciding on a level of
conformance (which could have an evaluation document all to itself) to "is
'topmargin' an accessibility hazard or harmless?"

Are focus groups/user testing groups used on these documents?

Thanks for your time,
Liz Roberts
Received on Tuesday, 30 October 2001 13:47:23 GMT

This archive was generated by hypermail 2.2.0+W3C-0.50 : Tuesday, 12 January 2010 00:13:08 GMT