Re: Input to automatic generation of test running dashboard

Thanks Felix. I will probably steal this idea for generating the website :).

Thanks,
Leroy


On 17 December 2012 21:10, Felix Sasaki <fsasaki@w3.org> wrote:

> Hi all,
>
> we discussed today the compilation of test results, see
> http://www.w3.org/2012/12/17-**mlw-lt-minutes.html#item03<http://www.w3.org/2012/12/17-mlw-lt-minutes.html#item03>
>
> In case it might help, below is an overview how we did that for ITS 1.0:
>
> 1) have a matster file that lists all output files, with a unique relation
> to an input file and to an implementation
> http://www.w3.org/**International/its/tests/input-**file-list.xml<http://www.w3.org/International/its/tests/input-file-list.xml>
> see e.g.
> <inputFile name="Translate1.xml" datacat="translate"
> relatedTest="translate-global-**embedded-rules">
>  <resultFile location="test1/Translate1-**result.xml"
> implementation="test1"/>
>  <resultFile location="test2/Translate1-**result.xml"
> implementation="test2"/>
>  <resultFile location="test3/Translate1-**result.xml"
> implementation="test3"/>
> </inputFile>
>
> 2) have a process
> http://www.w3.org/**International/its/tests/**compare.xsl<http://www.w3.org/International/its/tests/compare.xsl>
> that runs through input-file-list.xml and for each result file does a
> comparison with the expected output
> <xsl:call-template name="compareResult">
>  <xsl:with-param name="expectedResult" as="item()">
>   <xsl:copy-of select="doc(concat('expected/'**
> ,substring-before(../@name,'.**xml'),'-result.xml'))"/>
>  </xsl:with-param>
>  <xsl:with-param name="actualResult" as="item()">
>  <xsl:copy-of select="doc(string(@location))**"/>
>  </xsl:with-param>
>  <xsl:with-param name="implname">
>  <xsl:value-of select="@implementation"/>
> </xsl:with-param>
>  <xsl:with-param name="inputfilename"><xsl:**value-of
> select="../@name"/></xsl:with-**param>
>  <xsl:with-param name="relatedTest"><xsl:value-**of
> select="../@relatedTest"/></**xsl:with-param>
> </xsl:call-template>
>
> 3) the result of running that process is at
> http://tinyurl.com/its10-**testsuite-dashboard<http://tinyurl.com/its10-testsuite-dashboard>
> and also linked from
> http://www.w3.org/**International/its/tests/<http://www.w3.org/International/its/tests/>
> (see the "click here" link)
> then there is an additional stylsheet
> http://www.w3.org/**International/its/tests/**compareResultHtml.xsl<http://www.w3.org/International/its/tests/compareResultHtml.xsl>
> that generates the HTML table used in
> http://www.w3.org/**International/its/tests/<http://www.w3.org/International/its/tests/>
>
> The a bit tricky part of the process was in
> http://www.w3.org/**International/its/tests/**compare.xsl<http://www.w3.org/International/its/tests/compare.xsl>
> since we had the XML based output format and had to do a comparison of the
> XML files. With the text based format in the ITS 2.0 test suite a simple
> text based comparison should do it. But a file listing input from the
> various implementors like
> http://www.w3.org/**International/its/tests/input-**file-list.xml<http://www.w3.org/International/its/tests/input-file-list.xml>
> might be hepful. One could then auto-generate from this also the main test
> suite page
> https://github.com/finnle/ITS-**2.0-Testsuite/blob/master/**
> its2.0/its-testsuite.html<https://github.com/finnle/ITS-2.0-Testsuite/blob/master/its2.0/its-testsuite.html>
>
> Just some thoughts.
>
> Best,
>
> Felix
>
>
>