Re: Questions and proposals related to strawman schema from F2F

On Tue, 10 Sep 2002, Wendy A Chisholm wrote:
> 
> Ian, from your message it seems that severity is only intended to be used 
> with pass or fail and not with notTested, notApplicable, and 
> cannotTell.  Can you confirm this?

Sort-of.

notTested is a special case of lack of a result.

notApplicable is fail severity 0%.

cannotTell is pass or fail (depending on which is the 'default') with
confidence 0%. Whether it is pass or fail depends on the test, e.g. a test
which of stylesheet linking may test that a stylesheet is not linked in
certain error conditions, in which case you couldn't tell the difference
between the UA passing the test and the UA not supporting stylesheets, so
the result would be Pass with confidence 0%. (An outright fail would be
obvious as the stylesheet _would_ be linked.)

At least, that's how I store the data in my database: as either pass or
fail, with numeric severities and confidences.

In practice, though, there are only a certain number of states which can
occur in my system:

   1 passed completely                         PASS
   2 passed with unrelated errors              PASS
   3 passed partially                          PASS
   4 either passed or not implemented          CAN'T TELL
   5 not implemented at all                    NOT APPLICABLE
   6 either failed or not implemented          CAN'T TELL
   7 failed                                    FAIL
   8 failed so badly the feature is unusable   FAIL
   9 crash                                     FAIL

Each of these (especially 2, 3, 7 and 8) usually have associated free-form
comments explaining exactly what the error was.

So I would probably be happy simply using Charles' idea of scrapping
severity and confidence and explicitly calling out the 9 values. (10 if
you include the "NOT TESTED" non-value.)

As Charles said, the confidence issue can probably be out-sourced to some
other RDF vocabulary.

-- 
Ian Hickson                                      )\._.,--....,'``.    fL
"meow"                                          /,   _.. \   _\  ;`._ ,.
http://index.hixie.ch/                         `._.-(,_..'--(,_..'`-.;.'

Received on Saturday, 28 September 2002 12:01:04 UTC