Re: Proposal: severity axis on test result

On Sun, 30 Jun 2002, Charles McCathieNevile wrote:
>
>   Test results currently have a type (pass, fail, not applicable, etc) and a
>   confidence (high, medium, low).
>
>   I propose you add a third property: severity. This would be a record of how well
>   or badly the test passed or failed.

Agreed.  I've already added it to the test definition language I've
just been doodling with.

> I would propose that we have qualitative rather than quantitative ratings.
> For some use cases you Yb - pass with unrelated errors would count as a pass,
> and for some cases it would score as a fail. So we would need to know what
> they are.

I'd second that (I thought that was the original proposal as discussed in
the f2f?)


> The question also arises as to how many kinds of result we should include in
> earl and at what point we should leave people to subclass them for their own
> more detailed uses.

I wonder if we might want to use some syslog-like vocabulary,
CRITICAL-ALARM-ERROR-WARNING-NOTICE sort of thing within EARL?
That still leaves the way open for people to do their own thing.

-- 
Nick Kew

Available for contract work - Programming, Unix, Networking, Markup, etc.