RE: Test suite dashboard (Re: [All implementers] test sute master file)

Hi Felix,

Since I probably won't make it to the next call, I wanted to say that I totally agree with you about what is stated at point 1) because I think it's the best way to go, besides I checked it and it's really easy to understand and to interact with.
Per point 2) we will modify and adapt the output from now on.

Cheers,
Pablo.
>>>>>>>>>

Hi all again,

based on the latest version of
https://docs.google.com/spreadsheet/ccc?key=0AgIk0-aoSKOadG5HQmJDT2EybWVvVC1VbnF5alN2S3c#gid=0 

and on what is in github, I have created attached dashboard. It now contains a comparison of files that are in github.

Some thoughts below.

1) The generation of a machine readable file from
https://docs.google.com/spreadsheet/ccc?key=0AgIk0-aoSKOadG5HQmJDT2EybWVvVC1VbnF5alN2S3c#gid=0
is error prone. Could we give up the google docs and move to the following process:
- add the test suite master file TestStuiteMaster.xml to github
- ask implementers to update the file if they contribute new output, and to commit that output to github.
- Leroy or others would "only" update input files, or even better: 
implementers contributing these would do this themselves
- Re-generate testsuiteDashboard.html and testsuiteDashboard.xml. I could take care of that.

Since many people are on holiday I'd propose to decide about these things and the test suite master on the 7 January call.

2) Current state of test suite
See testsuiteDashboard.html
Thanks mostly to Enlaso, we have 30% coverage of tests - a big "thanks" 
to Yves and Fredrik. But many others contributed a lot too; there are only minor errors in the files. E.g. Linguaserve deleted the index of an element if there is only one element, e.g.
a) /html/head/body/p[1]
but the reference output files do this only for the root element, e.g.
b) /html/head[1]/body[1]/p[1]
Either solution is fine, but since I think most implementers did use version a), maybe Linguaserve can change the output? That is, no index only at the root element, for all other elements there is an index.
I think my "highly optimization-oriented mind"
http://lists.w3.org/Archives/Public/public-multilingualweb-lt-tests/2012Dec/0171.html
is the reason for the misunderstanding :)

3) What's needed next
We need output files in github.

Best,

Felix

Am 24.12.12 11:05, schrieb Felix Sasaki:
> Hi all again,
>
> based on the below XML export from
> https://docs.google.com/spreadsheet/ccc?key=0AgIk0-aoSKOadG5HQmJDT2Eyb
> WVvVC1VbnF5alN2S3c#gid=0
>
> I have generated a first version of a test suite dashboard, see 
> attachment. Currently it contains only the commitments made in the 
> google docs, not what is actually in github. I will update this during 
> the end of year season so that the implementers github files are 
> actually loaded and compared to the expected output.
>
> Best,
>
> Felix
>
> Am 21.12.12 13:18, schrieb Felix Sasaki:
>> Hi all,
>>
>> Out of on XML export from the spreadsheet I have generated a test 
>> suite "master file", see attachment and the discussion at 
>> http://lists.w3.org/Archives/Public/public-multilingualweb-lt-tests/2
>> 012Dec/0148.html
>>
>>
>> Please don't change the styling of the spreadsheet or the general 
>> structure, since the re-generation of the master file relies on these.
>>
>> Could you give feedback on two aspects:
>>
>> 1) Structure of and information in the master file - is anything 
>> missing, too much etc.
>>
>> 2) Misalignment with
>> https://docs.google.com/spreadsheet/ccc?key=0AgIk0-aoSKOadG5HQmJDT2Ey
>> bWVvVC1VbnF5alN2S3c#gid=0
>>
>> In above spreadsheets, several cells show green. But the output data 
>> is not on github, or it is only partially on github. So we have 
>> reached milestone 2 in the spreadsheet, but it's not yet reflected in 
>> github. Can you check "your" files and push them to github?
>>
>> Thanks,
>>
>> Felix
>

Received on Friday, 28 December 2012 11:43:28 UTC