Re: Discussion: WCAG Test Tool Behaviour

I think it is best to give info on the "untested" points - probably 
not as a "nag" as you put it, but as a short summary of the other 
issues people need to be aware of. It wouldn't have to be part of 
every set of test results.

People using your tool may not be aware of the "untestable" WCAG 
items, and if using your tool is the ONLY thing they do to "cover" 
accessibility, they may never find out, unless the tool alerts them 
to the issues.

Cheers

Rebecca Cox


At 12:09 AM +0000 19/3/02, Nick Kew wrote:
>I've had a comment regarding the way Page Valet reports WCAG
>accessibility warnings.  My correspondent[1] thinks it should
>mention any WCAG points that are untested, as this is how some
>other accessibility tools work and it makes Valet seems less complete[2].
>
>As it stands, Page Valet will generate warnings when it detects an
>error (or potential error).  When it applies a test that is passed
>it says nothing.  When it doesn't apply a test, it says nothing.
>
>Now the suggestion is that is should - conceptually - test every point
>in the WCAG.  In cases like #14.1/etc that self-evidently can't be
>tested by any tool, it should issue a nag message - whatever page
>is being tested.
>
>Now, I have several reasons not to apply this kind of completeness.
>But maybe I'm missing the wider view.  Comments, please, on:
>
>* Nag messages are a turnoff.  If more than a couple of messages
>   are repeated in every evaluation, users will rapidly learn to
>   ignore them.
>
>* If we say that Valet should exhaustively list every point in WCAG,
>   are we not saying that WCAG itself is the best test tool - and the
>   only one that is never wrong?
>
>* Valet applies tests to every element and attribute, and displays
>   messages in place in the source code.  This is designed to make it
>   more useful to its users:
>	"there is an error right here"
>   as opposed to
>	"there is an error somewhere"
>   Listing tests per-document is an entirely different approach.
>
>* Because Valet applies per-element tests, there are tests that
>   are both applied and not applied.  For example, #4.1/#4.3 (the lang
>   attribute) is tested for the <html> element[3], but not for other
>   elements.  Yet to be exhaustive, we should test *every* element,
>   lest it contain a passage in another language.  For the tool to
>   be manageable at all, it needs to make compromises.
>
>* Bottom line: too many nag warnings will dilute the overall message
>   of the tests - like useful content getting buried in waffle or
>   (in our times) spam.
>
>
>Now, one possible solution here is to retain the present scheme
>(modulo updates) for the human-readable HTML reports, but to include
>the additional nag messages in the machine-readable EARL reports
>so they get into databases populated by Valet (such as W3C's
>Annotea).
>
>
>[1] She reads this list, so she'll doubtless reply if I'm
>     misrepresenting her.
>[2] Section508 testing does take this more legalistic nagging approach.
>     That works because 508 is much briefer, and expressed in terms that
>     are easier to incorporate into an automatic tool than WCAG.
>[3] The lang attribute for html shouldn't be necessary at all in the
>     presence of an HTTP Content-Language header.  But testing for it
>     seems to be a requirement for any WCAG testing tool.
>
>--
>Nick Kew
>
>Site Valet - the mark of Quality on the Web.
><URL:http://valet.webthing.com/>

Received on Tuesday, 19 March 2002 15:34:49 UTC