Discussion: WCAG Test Tool Behaviour

I've had a comment regarding the way Page Valet reports WCAG
accessibility warnings.  My correspondent[1] thinks it should
mention any WCAG points that are untested, as this is how some
other accessibility tools work and it makes Valet seems less complete[2].

As it stands, Page Valet will generate warnings when it detects an
error (or potential error).  When it applies a test that is passed
it says nothing.  When it doesn't apply a test, it says nothing.

Now the suggestion is that is should - conceptually - test every point
in the WCAG.  In cases like #14.1/etc that self-evidently can't be
tested by any tool, it should issue a nag message - whatever page
is being tested.

Now, I have several reasons not to apply this kind of completeness.
But maybe I'm missing the wider view.  Comments, please, on:

* Nag messages are a turnoff.  If more than a couple of messages
  are repeated in every evaluation, users will rapidly learn to
  ignore them.

* If we say that Valet should exhaustively list every point in WCAG,
  are we not saying that WCAG itself is the best test tool - and the
  only one that is never wrong?

* Valet applies tests to every element and attribute, and displays
  messages in place in the source code.  This is designed to make it
  more useful to its users:
	"there is an error right here"
  as opposed to
	"there is an error somewhere"
  Listing tests per-document is an entirely different approach.

* Because Valet applies per-element tests, there are tests that
  are both applied and not applied.  For example, #4.1/#4.3 (the lang
  attribute) is tested for the <html> element[3], but not for other
  elements.  Yet to be exhaustive, we should test *every* element,
  lest it contain a passage in another language.  For the tool to
  be manageable at all, it needs to make compromises.

* Bottom line: too many nag warnings will dilute the overall message
  of the tests - like useful content getting buried in waffle or
  (in our times) spam.


Now, one possible solution here is to retain the present scheme
(modulo updates) for the human-readable HTML reports, but to include
the additional nag messages in the machine-readable EARL reports
so they get into databases populated by Valet (such as W3C's
Annotea).


[1] She reads this list, so she'll doubtless reply if I'm
    misrepresenting her.
[2] Section508 testing does take this more legalistic nagging approach.
    That works because 508 is much briefer, and expressed in terms that
    are easier to incorporate into an automatic tool than WCAG.
[3] The lang attribute for html shouldn't be necessary at all in the
    presence of an HTTP Content-Language header.  But testing for it
    seems to be a requirement for any WCAG testing tool.

-- 
Nick Kew

Site Valet - the mark of Quality on the Web.
<URL:http://valet.webthing.com/>

Received on Monday, 18 March 2002 19:10:53 UTC