W3C home > Mailing lists > Public > xproc-dev@w3.org > December 2008

Re: Submitting test suite results to tests.xproc.org

From: David A. Lee <dlee@calldei.com>
Date: Fri, 5 Dec 2008 12:04:37 -0500
Message-ID: <3628808A290540238F6C313870733614@calldei.com>
To: "Norman Walsh" <ndw@nwalsh.com>, "XProc Dev" <xproc-dev@w3.org>

This is really great stuff ... I cant wait to start contributing ...
( although right now I'm at 0 passes ... maybe 1 by end of day  :)

Suggestion.  For future usability and comparisons I'd really like to see 
some kind of performance metrics in these reports.  We all know absolute 
numbers are misleading at beast, but if interpreted with some wisdom they 
can be useful.

Example:  In the <processor> section (or a new optional section) a section 
for the environment run on such as

CPU (maybe with some optional sub-details  for things like cpu vendor, disk 
size, number of CPU's etc)
OS
RAM

Then for each test the realtime for the test execution (but of course not 
the results collection).

For now these would be prety meaningless as the tests are almost all very 
small.
But as we create more tests that are more complex the numbers would be very 
useful.


Alternatively ... maybe this is more valuable as a seperate report entirely 
with performance benchmarks as the goal not spec compliance.

-David

-----------------------------------------------------------
David A. Lee
dlee@calldei.com
http://www.calldei.com
Received on Friday, 5 December 2008 17:05:22 GMT

This archive was generated by hypermail 2.2.0+W3C-0.50 : Friday, 5 December 2008 17:05:23 GMT