SV: "Bug" categorization [Was: Minutes in brief and action items]

comments inlined

-----Ursprungligt meddelande-----
Från: Arnold, Curt [mailto:Curt.Arnold@hyprotech.com]
Skickat: den 1 juni 2001 19:53
Till: 'www-dom-ts@w3.org'
Ämne: "Bug" categorization [Was: Minutes in brief and action items]


For comparison, you might look at Apache's bugzilla:

http://nagoya.apache.org/bugzilla/query.cgi

I would have actually flipped those definitions of Category and Group.
However both are vague enough that they can mean anything until you
populate the acceptable values.

I would have had Category roughly equivalent to Severity and 
Group roughly equivalent to Component.  I'll continue the discussion
using the Apache terms, but eventually we would want to bind
them to one of to Category and Group.

The Apache bugzilla has the following values for Severity:
Blocker, Critical, Major, Normal, Minor, Enhancement.

That is reasonably for software components, but to describe
submitted tests and issues against them, we would want
to have some severities like:

invalid: The test is not valid against the schema for tests plus 
additional constraints such as declaring variables before use.

[dd] yes

optional-behavior: The test checks behavior beyond that required by the
spec.  
For example, requiring entity references when the spec doesn't require them.

[dd] could you give an example so that I can be sure of understanding?

misinterpretation: The test misinterprets the DOM spec.

[dd] I think this falls into two different categories: the test being
refuted because of misinterpretation(s), and the test being put on hold
while being investigated be the DOM WG and discussed on the mailing list

brittle: The test is fails when used with different processor settings (such
as ignore element white space), doesn't convert well to other languages.

[dd] wouldn't this indicate that the implementation just fails? or woudl it
indicate that the test is in need of enhancement?

enhancement: The test could make some additional assertions that would add
value.

[dd] yes. this could be optional, though, since we wouldn't touch submitted
tests, but we could discuss it so that the submitter could alter it.
however, this is different from the way I use 'enhancement' in connection to
'brittle' tests.

Any other ways a test could be bad (or at least sub optimal)?

[dd] not bad or suboptimal, but 'while being discussed' (concerning
interpretation, for example) could be a valuable category (it would have to
be lifted out of the suite) to be discussed and perhaps only then be put in
the misinterpretation category


On Component, using the names of major test submissions might be
appropriate.
For example, nist.dom1.attr, nist.html1.HTMLTableElement, or
xmlconf.dom2.events.EventTarget.  The transforms, schemas, etc could be
components but use the standard software severity.

[dd] sounds fine to me. how do we tag tests that persons have submitted?
names?

Now I know what the trashcan next to developer name does.

Received on Friday, 1 June 2001 15:25:49 UTC