Format of implementation compliance test report

Dear all

We're still looking into SOAP/JMS compliance testing, and - in anticipation
of starting testing -  we would like some guidance on the test report that
we'd need to produce.

I've searched the W3C site, but have not found many examples of
implementation conformance reports.  Is there a standard W3C template?  If
not, I'd like to propose that the SOAP/JMS working group use a format based
on the Implementation Report Template for CSS2.1 Test Suite [1].

The report would be a simple text document, with a hash (pound) character
denoting a comment to the end of the line, and tab-separated data.  It
would be formatted as follows :

   The first line is a comment with the implementation name, version, and
OS (separated by commas)
   The second line is a comment with the date of the test report in
YYYY/MM/DD format
   The third line is a comment with the SOAP/JMS test suite's base URL [2]
   The fourth line is a literal "testname\tresult"
   The rest of the lines give the test's anchor name (within the base URL
document), followed by one or more tabs, and the result (which may be one
of : pass fail na invalid ? )
   Blank lines and other comments are ignored.

Example:

# SOAP/JMS implementation name, version, OS
# YYYY/MM/DD
# http://dev.w3.org/2008/ws/soapjms/testcases/testcases/testcases.html
testname    result
test0001	?
test0002	?
test0101	?
test0102	?
      ...
test1109	?


Any thoughts / comments?

Many thanks
Mark

[1]
http://lists.w3.org/Archives/Public/public-css-testsuite/2010Aug/0020.html
[2] http://dev.w3.org/2008/ws/soapjms/testcases/testcases/testcases.html

Received on Tuesday, 1 March 2011 21:08:11 UTC