RE: Submitting test suite results to

> Dear implementors,
> I've cooked up an XML format for reporting test suite 
> results. It's defined
> by the schema at 
> (or .rng).
> (Alas, there's no documentation yet.)

Great news! But the schemas still don't seem to be there (nor .rnc nor

> For an example of the format, you can download the XML Calabash report
> From

I don't think it's the best example of the format, it contains too few
failures :)

> I'm open to suggestions for changes/improvements.

I think that for the tests that are expected to fail, the schema should
distinguish between three possible cases:

- processor did not fail
- pricessor did fail with proper error code
- processor failed, but with a different error code (and interpret this
as 50% success or something like that?)

(But perhaps the schema already takes this into account, I can't check.)

> As an added incentive to use the format :-), I've created a web
> service that will allow you to submit results in that format to
> and have the results posted immediately. Like
> the XML Calabash report at
> Send me the short name that you would like to use to identify your
> implementation and a password that you'd like and I'll set it up.
> The short name will appear in the URI:

I will submit the results as soon as I implement support for the schema.

> And finally, here's the pipeline that I use to run the tests and
> submit the results. It's actually a fairly interesting pipeline.
> Questions and/or suggestions for improvements most welcome.
> <p:declare-step xmlns:p=""
> 		xmlns:c=""
> 		xmlns:cx=""
> 		exclude-inline-prefixes="c cx"
> 		name="main">
> </p:declare-step>

Indeed interesting. Maybe it is not a bad idea to post this pipeline on as the preferred way of submitting the results :)

Once again, thanks for the effort.


Received on Thursday, 4 December 2008 13:15:40 UTC