Submitting test suite results to tests.xproc.org

Dear implementors,

I've cooked up an XML format for reporting test suite results. It's defined
by the schema at http://tests.xproc.org/schemas/testreport.rnc (or .rng).
(Alas, there's no documentation yet.)

For an example of the format, you can download the XML Calabash report
From http://tests.xproc.org/results/calabash/report.xml

I'm open to suggestions for changes/improvements.

As an added incentive to use the format :-), I've created a web
service that will allow you to submit results in that format to
tests.xproc.org and have the results posted immediately. Like
the XML Calabash report at http://tests.xproc.org/results/calabash/

Send me the short name that you would like to use to identify your
implementation and a password that you'd like and I'll set it up.

The short name will appear in the URI:

  http://tests.xproc.org/results/<shortname>

If you don't want to make the results of public, but you want to
submit them to the WG, let me know and I'll setup something for that
(but first, I'll try to persuade you to publish your results!).

And finally, here's the pipeline that I use to run the tests and
submit the results. It's actually a fairly interesting pipeline.
Questions and/or suggestions for improvements most welcome.

<p:declare-step xmlns:p="http://www.w3.org/ns/xproc"
		xmlns:c="http://www.w3.org/ns/xproc-step"
		xmlns:cx="http://xmlcalabash.com/ns/extensions"
		exclude-inline-prefixes="c cx"
		name="main">
<p:output port="result"/>
<p:option name="host" select="'tests.xproc.org'"/>
<p:option name="username" select="'calabash'"/>
<p:option name="password" select="''"/>

<p:documentation xmlns="http://www.w3.org/1999/xhtml">
<div>
  <p>This pipeline runs the XProc test suite and reports the results.</p>
  <p>It has three options: <code>host</code>, <code>username</code>, and
  <code>password</code>.</p>
  <p>The <code>host</code> option is used to specify the host to use
  in the URIs of test suite documents. (Because Norm has a local copy of
  <a href="http://tests.xproc.org/">http://tests.xproc.org/</a>.)</p>
  <p>The <code>username</code> and <code>password</code> are used to submit
  the test results to <a href="http://tests.xproc.org/">http://tests.xproc.org</a>.
  If run without a password, it simply outputs the report document.</p>
</div>
</p:documentation>

<p:string-replace match="uri/text()">
  <p:input port="source">
    <p:inline>
      <test-suites>
	<uri>/tests/required/test-suite.xml</uri>
	<uri>/tests/optional/test-suite.xml</uri>
	<uri>/tests/extension/test-suite.xml</uri>
      </test-suites>
    </p:inline>
  </p:input>
  <p:with-option name="replace"
		 select="concat('concat(&quot;http://',$host,'&quot;,.)')">
    <p:empty/>
  </p:with-option>
</p:string-replace>

<p:exec command="/projects/src/calabash/runtests">
  <p:input port="source">
    <p:empty/>
  </p:input>
  <p:with-option name="args" select="string-join(//uri,' ')"/>
</p:exec>

<p:choose>
  <p:when test="$password = ''">
    <p:identity/>
  </p:when>
  <p:otherwise>
    <p:wrap wrapper="c:body" match="/">
      <p:input port="source" select="/c:result/*"/>
    </p:wrap>

    <p:add-attribute match="c:body"
		     attribute-name="content-type"
		     attribute-value="application/xml"/>

    <p:wrap wrapper="c:request" match="/"/>

    <p:add-attribute attribute-name="password" match="/c:request">
      <p:with-option name="attribute-value" select="$password"/>
    </p:add-attribute>

    <p:add-attribute attribute-name="username" match="/c:request">
      <p:with-option name="attribute-value" select="$username"/>
    </p:add-attribute>

    <p:set-attributes match="c:request">
      <p:input port="attributes">
	<p:inline>
	  <c:request href="http://tests.xproc.org/results/submit/report"
		     method="post" auth-method="Basic"
		     send-authorization="true"/>
	</p:inline>
      </p:input>
    </p:set-attributes>

    <p:http-request/>
  </p:otherwise>
</p:choose>

</p:declare-step>

                                        Be seeing you,
                                          norm

-- 
Norman Walsh <ndw@nwalsh.com> | Men are more like the times they live
http://nwalsh.com/            | in than they are like their
                              | fathers.--Ali Ibn-abi-talib