Test suite dashboard (Re: [All implementers] test sute master file)

Hi all again,

based on the latest version of
https://docs.google.com/spreadsheet/ccc?key=0AgIk0-aoSKOadG5HQmJDT2EybWVvVC1VbnF5alN2S3c#gid=0 

and on what is in github, I have created attached dashboard. It now 
contains a comparison of files that are in github.

Some thoughts below.

1) The generation of a machine readable file from
https://docs.google.com/spreadsheet/ccc?key=0AgIk0-aoSKOadG5HQmJDT2EybWVvVC1VbnF5alN2S3c#gid=0
is error prone. Could we give up the google docs and move to the 
following process:
- add the test suite master file TestStuiteMaster.xml to github
- ask implementers to update the file if they contribute new output, and 
to commit that output to github.
- Leroy or others would "only" update input files, or even better: 
implementers contributing these would do this themselves
- Re-generate testsuiteDashboard.html and testsuiteDashboard.xml. I 
could take care of that.

Since many people are on holiday I'd propose to decide about these 
things and the test suite master on the 7 January call.

2) Current state of test suite
See testsuiteDashboard.html
Thanks mostly to Enlaso, we have 30% coverage of tests - a big "thanks" 
to Yves and Fredrik. But many others contributed a lot too; there are 
only minor errors in the files. E.g. Linguaserve deleted the index of an 
element if there is only one element, e.g.
a) /html/head/body/p[1]
but the reference output files do this only for the root element, e.g.
b) /html/head[1]/body[1]/p[1]
Either solution is fine, but since I think most implementers did use 
version a), maybe Linguaserve can change the output? That is, no index 
only at the root element, for all other elements there is an index.
I think my "highly optimization-oriented mind"
http://lists.w3.org/Archives/Public/public-multilingualweb-lt-tests/2012Dec/0171.html
is the reason for the misunderstanding :)

3) What's needed next
We need output files in github.

Best,

Felix

Am 24.12.12 11:05, schrieb Felix Sasaki:
> Hi all again,
>
> based on the below XML export from
> https://docs.google.com/spreadsheet/ccc?key=0AgIk0-aoSKOadG5HQmJDT2EybWVvVC1VbnF5alN2S3c#gid=0 
>
> I have generated a first version of a test suite dashboard, see 
> attachment. Currently it contains only the commitments made in the 
> google docs, not what is actually in github. I will update this during 
> the end of year season so that the implementers github files are 
> actually loaded and compared to the expected output.
>
> Best,
>
> Felix
>
> Am 21.12.12 13:18, schrieb Felix Sasaki:
>> Hi all,
>>
>> Out of on XML export from the spreadsheet I have generated a test 
>> suite "master file", see attachment and the discussion at
>> http://lists.w3.org/Archives/Public/public-multilingualweb-lt-tests/2012Dec/0148.html 
>>
>>
>> Please don't change the styling of the spreadsheet or the general 
>> structure, since the re-generation of the master file relies on these.
>>
>> Could you give feedback on two aspects:
>>
>> 1) Structure of and information in the master file - is anything 
>> missing, too much etc.
>>
>> 2) Misalignment with
>> https://docs.google.com/spreadsheet/ccc?key=0AgIk0-aoSKOadG5HQmJDT2EybWVvVC1VbnF5alN2S3c#gid=0 
>>
>> In above spreadsheets, several cells show green. But the output data 
>> is not on github, or it is only partially on github. So we have 
>> reached milestone 2 in the spreadsheet, but it's not yet reflected in 
>> github. Can you check "your" files and push them to github?
>>
>> Thanks,
>>
>> Felix
>