Re: browsable test results

Dan Connolly wrote:
>...How much of the wheel have we reinvented?

From the point of view of the OASIS test case management system used
for XSLT, not too much. At a quick glance, I think the two are
compatible.

From the point of view of QAWG guidelines, I see a problem in the
result table where you report that a product has passed 100% of the
test cases for each group. The string "No failures found" would be
more appropriate. The issues about percentages include:
1. Implication that the current suite is 100% of all the tests that
   should be there. Test suites that are being expanded frequently
   won't have a stable notion of 100%.
2. Implication that high numbers under 100% are pretty good. Each
   class of product may have its own notion of how seriously interop
   has been hurt by a score of, say, 96%.
3. Implication that all tests count equally. Product X's 96% might
   be much worse than Product Y's 96%, depending on the cases that
   comprise the failing 4% on each.
.................David Marston