Re: Input to automatic generation of test running dashboard

I would agree. I also think an online and downloadable file differ, customised for ITS 2.0 would be good for our public test suite website. We'll look into this in the new year. 

Dom

On 18 Dec 2012, at 11:40, Leroy Finn <finnle@tcd.ie> wrote:

> Thanks Felix. I will probably steal this idea for generating the website :).
> 
> Thanks,
> Leroy
> 
> 
> On 17 December 2012 21:10, Felix Sasaki <fsasaki@w3.org> wrote:
> Hi all,
> 
> we discussed today the compilation of test results, see
> http://www.w3.org/2012/12/17-mlw-lt-minutes.html#item03
> 
> In case it might help, below is an overview how we did that for ITS 1.0:
> 
> 1) have a matster file that lists all output files, with a unique relation to an input file and to an implementation
> http://www.w3.org/International/its/tests/input-file-list.xml
> see e.g.
> <inputFile name="Translate1.xml" datacat="translate" relatedTest="translate-global-embedded-rules">
>  <resultFile location="test1/Translate1-result.xml" implementation="test1"/>
>  <resultFile location="test2/Translate1-result.xml" implementation="test2"/>
>  <resultFile location="test3/Translate1-result.xml" implementation="test3"/>
> </inputFile>
> 
> 2) have a process
> http://www.w3.org/International/its/tests/compare.xsl
> that runs through input-file-list.xml and for each result file does a comparison with the expected output
> <xsl:call-template name="compareResult">
>  <xsl:with-param name="expectedResult" as="item()">
>   <xsl:copy-of select="doc(concat('expected/',substring-before(../@name,'.xml'),'-result.xml'))"/>
>  </xsl:with-param>
>  <xsl:with-param name="actualResult" as="item()">
>  <xsl:copy-of select="doc(string(@location))"/>
>  </xsl:with-param>
>  <xsl:with-param name="implname">
>  <xsl:value-of select="@implementation"/>
> </xsl:with-param>
>  <xsl:with-param name="inputfilename"><xsl:value-of select="../@name"/></xsl:with-param>
>  <xsl:with-param name="relatedTest"><xsl:value-of select="../@relatedTest"/></xsl:with-param>
> </xsl:call-template>
> 
> 3) the result of running that process is at
> http://tinyurl.com/its10-testsuite-dashboard
> and also linked from
> http://www.w3.org/International/its/tests/
> (see the "click here" link)
> then there is an additional stylsheet
> http://www.w3.org/International/its/tests/compareResultHtml.xsl
> that generates the HTML table used in
> http://www.w3.org/International/its/tests/
> 
> The a bit tricky part of the process was in
> http://www.w3.org/International/its/tests/compare.xsl
> since we had the XML based output format and had to do a comparison of the XML files. With the text based format in the ITS 2.0 test suite a simple text based comparison should do it. But a file listing input from the various implementors like
> http://www.w3.org/International/its/tests/input-file-list.xml
> might be hepful. One could then auto-generate from this also the main test suite page
> https://github.com/finnle/ITS-2.0-Testsuite/blob/master/its2.0/its-testsuite.html
> 
> Just some thoughts.
> 
> Best,
> 
> Felix
> 
> 
>

Received on Tuesday, 18 December 2012 11:45:24 UTC