Preliminary results from ER Test Files And Tool

Chris and Josh,

This is great!  I've begun a preliminary review of SSB Technologies InSight 
(version 2.4 - couldn't download the latest - I'll have to talk with Tim 
again).  The results that I have so far are at: 
http://www.w3.org/WAI/ER/2001/07/insight.rdf

In the quick 10 tests that I did, I found some interesting errors in InSight.

I have a few questions/suggestions about the ATR tool and test files.

1. If the tool being evaluated identifies the problem, but doesn't offer 
the suggestion, does it pass?  For example, test file 1-1-1-f12 is a 
horizontal rule image without alt-text.  InSight correctly identifies that 
it is missing alt-text but doesn't suggest "horizontal rule."  The ATR tool 
says, "Should trigger:  Image missing ALT text (suggest "horizontal rule").

I said that it passed it, since it did trigger.  Perhaps these ought to be 
separate - the suggestion from the error detection.  I think it would be 
good to know if the correct suggestion was made or if any suggestion was made.


2. Several of the test files do not say what should trigger.  e.g., 
1-1-10-f1, 1-1-12-f1, 1-1-14-f1, 1-1-14-f2, 1-1-14-f3, 1-1-14-f4

1-1-10-f1.xml  has a comment element while 1-1-1-f1.xml has a 
suggested_message and an empty comment element.


3.  During my review I made comments about additional messages that InSight 
was generating as I went along.  Those comments are not recorded in the 
results.  Also, the comment field did not clear when I selected a new test 
file (as I had expected it to do).  To write something new I had to delete 
the text then write.  BUT none of that was captured.


Thanks for your work Chris and Josh!
--wendy
--
wendy a chisholm
world wide web consortium
web accessibility initiative
seattle, wa usa
/--

Received on Thursday, 2 August 2001 11:02:43 UTC