Re: Submitting test suite results to

This is really great stuff ... I cant wait to start contributing ...
( although right now I'm at 0 passes ... maybe 1 by end of day  :)

Suggestion.  For future usability and comparisons I'd really like to see 
some kind of performance metrics in these reports.  We all know absolute 
numbers are misleading at beast, but if interpreted with some wisdom they 
can be useful.

Example:  In the <processor> section (or a new optional section) a section 
for the environment run on such as

CPU (maybe with some optional sub-details  for things like cpu vendor, disk 
size, number of CPU's etc)

Then for each test the realtime for the test execution (but of course not 
the results collection).

For now these would be prety meaningless as the tests are almost all very 
But as we create more tests that are more complex the numbers would be very 

Alternatively ... maybe this is more valuable as a seperate report entirely 
with performance benchmarks as the goal not spec compliance.


David A. Lee

Received on Friday, 5 December 2008 17:05:22 UTC