Re: Submitting test suite results to tests.xproc.org

This is really great stuff ... I cant wait to start contributing ...
( although right now I'm at 0 passes ... maybe 1 by end of day  :)

Suggestion.  For future usability and comparisons I'd really like to see 
some kind of performance metrics in these reports.  We all know absolute 
numbers are misleading at beast, but if interpreted with some wisdom they 
can be useful.

Example:  In the <processor> section (or a new optional section) a section 
for the environment run on such as

CPU (maybe with some optional sub-details  for things like cpu vendor, disk 
size, number of CPU's etc)
OS
RAM

Then for each test the realtime for the test execution (but of course not 
the results collection).

For now these would be prety meaningless as the tests are almost all very 
small.
But as we create more tests that are more complex the numbers would be very 
useful.


Alternatively ... maybe this is more valuable as a seperate report entirely 
with performance benchmarks as the goal not spec compliance.

-David

-----------------------------------------------------------
David A. Lee
dlee@calldei.com
http://www.calldei.com

Received on Friday, 5 December 2008 17:05:22 UTC