Re: Test harness results xml format

On 07/27/2010 03:28 AM, Kris Krueger wrote:
> As discussed at the last meeting this is the format of what the test harness could expose as xml.
> This could be used to  build an implementation report, if you have feedback (add data, remove data) please speak up.

> <?xml version="1.0" encoding="utf-8"?>
> <testRun version="1.0">
>    <userAgent>9.0.101101</userAgent>
>    <browserName>Internet Explorer</browserName>
>    <Date>07-26-2010</Date>
>    <Submitter>Microsoft Test Team</Submitter>
>    <targetTestSuiteName>W3C HTML5 Test Suite Ver 1.0</targetTestSuiteName>
>    <targetTestSuiteID>W3CHTML5-10</targetTestSuiteID>
>    <Tests>
>      <Test Uri="/w3c/tests/html5/Chapt2/sect2.1.1.3">
>        <featureName>Canvas</featureName>
>        <testName>Object literal - Get Set property</testName>
>        <specReference>7.2, 7.5</specReference>
>        <Result>Pass|Fail|Not Implemented</Result>
>      </Test>
>      <Test Uri="/w3c/tests/html5/Chapt2/sect2.1.1.4">
>        <featureName>Canvas</featureName>
>        <testName>Object literal - Get Set property</testName>
>        <specReference>7.2, 7.5</specReference>
>        <Result>Pass|Fail|Not Implemented</Result>
>      </Test>
>    </Tests>
> </testRun>

I'm not sure what the use cases / requirements we are trying to address 
here are, so it is impossible for me to judge what is needed from the 
above. That said, I would a-priori prefer a simpler json format, like 
(in some mixture of json and pseudo-bnf):

{
"ua":ua-id,
"results":{(test-id:result)*}
}

where

ua-id := user agant id string
test-id := path to test
result := "pass" | "fail"

I'm not sure what the point of "not implemented" as a separate state is. 
Since we don't really have optional features, "not implemented" is 
equivalent to "fail".

Received on Tuesday, 27 July 2010 08:55:51 UTC