Re: Discussion: WCAG Test Tool Behaviour

Yeah, I think in a user interface it is helpful to be clear about what hasn't
been tested - especially in something that is basically involved in testing.

(By the way, there are strategies for automating things like simple language
testing and repair - at least to the point of having an interactive
machine-supported process. Wanna challenge? ;-)

cheers

Chaals

On Wed, 20 Mar 2002, Rebecca Cox wrote:

  I think it is best to give info on the "untested" points - probably
  not as a "nag" as you put it, but as a short summary of the other
  issues people need to be aware of. It wouldn't have to be part of
  every set of test results.

  People using your tool may not be aware of the "untestable" WCAG
  items, and if using your tool is the ONLY thing they do to "cover"
  accessibility, they may never find out, unless the tool alerts them
  to the issues.

  Cheers

  Rebecca Cox


  At 12:09 AM +0000 19/3/02, Nick Kew wrote:
  >I've had a comment regarding the way Page Valet reports WCAG
  >accessibility warnings.  My correspondent[1] thinks it should
  >mention any WCAG points that are untested, as this is how some
  >other accessibility tools work and it makes Valet seems less complete[2].
  >
  >As it stands, Page Valet will generate warnings when it detects an
  >error (or potential error).  When it applies a test that is passed
  >it says nothing.  When it doesn't apply a test, it says nothing.
  >
  >Now the suggestion is that is should - conceptually - test every point
  >in the WCAG.  In cases like #14.1/etc that self-evidently can't be
  >tested by any tool, it should issue a nag message - whatever page
  >is being tested.
  >
  >Now, I have several reasons not to apply this kind of completeness.
  >But maybe I'm missing the wider view.  Comments, please, on:
  >
  >* Nag messages are a turnoff.  If more than a couple of messages
  >   are repeated in every evaluation, users will rapidly learn to
  >   ignore them.
  >
  >* If we say that Valet should exhaustively list every point in WCAG,
  >   are we not saying that WCAG itself is the best test tool - and the
  >   only one that is never wrong?
  >
  >* Valet applies tests to every element and attribute, and displays
  >   messages in place in the source code.  This is designed to make it
  >   more useful to its users:
  >	"there is an error right here"
  >   as opposed to
  >	"there is an error somewhere"
  >   Listing tests per-document is an entirely different approach.
  >
  >* Because Valet applies per-element tests, there are tests that
  >   are both applied and not applied.  For example, #4.1/#4.3 (the lang
  >   attribute) is tested for the <html> element[3], but not for other
  >   elements.  Yet to be exhaustive, we should test *every* element,
  >   lest it contain a passage in another language.  For the tool to
  >   be manageable at all, it needs to make compromises.
  >
  >* Bottom line: too many nag warnings will dilute the overall message
  >   of the tests - like useful content getting buried in waffle or
  >   (in our times) spam.
  >
  >
  >Now, one possible solution here is to retain the present scheme
  >(modulo updates) for the human-readable HTML reports, but to include
  >the additional nag messages in the machine-readable EARL reports
  >so they get into databases populated by Valet (such as W3C's
  >Annotea).
  >
  >
  >[1] She reads this list, so she'll doubtless reply if I'm
  >     misrepresenting her.
  >[2] Section508 testing does take this more legalistic nagging approach.
  >     That works because 508 is much briefer, and expressed in terms that
  >     are easier to incorporate into an automatic tool than WCAG.
  >[3] The lang attribute for html shouldn't be necessary at all in the
  >     presence of an HTTP Content-Language header.  But testing for it
  >     seems to be a requirement for any WCAG testing tool.
  >
  >--
  >Nick Kew
  >
  >Site Valet - the mark of Quality on the Web.
  ><URL:http://valet.webthing.com/>


-- 
Charles McCathieNevile    http://www.w3.org/People/Charles  phone: +61 409 134 136
W3C Web Accessibility Initiative     http://www.w3.org/WAI  fax: +33 4 92 38 78 22
Location: 21 Mitchell street FOOTSCRAY Vic 3011, Australia
(or W3C INRIA, Route des Lucioles, BP 93, 06902 Sophia Antipolis Cedex, France)

Received on Tuesday, 26 March 2002 02:40:32 UTC