Re: Test suite specs

Great, thanks for the info, Leroy.

Best,

Felix

2012/10/24 Leroy Finn <finnle@tcd.ie>

> Hey Felix,
>
> Yeah i do have a script that does it to all the files in this manner. If
> you and Yves think alphabetical is best then i will change the output
> to reflect this but currently i am reading through the new draft and
> getting the new test files up on the site. After the Lyon meeting I will
> update the output to alphabetical as i am producing new output anyway for
> the new files.
>
> Thanks,
> Leroy
>
>
> On 24 October 2012 22:07, Felix Sasaki <fsasaki@w3.org> wrote:
>
>> Hi Leroy, all,
>>
>> having a batch file to re-produce the output can indeed save a lot of
>> time. For ITS 1.0 we used such a setup, based on a "master file"
>> http://www.w3.org/International/its/tests/input-file-list.xml
>> that generates the HTML overview document
>> http://www.w3.org/International/its/tests/
>> and the expected results. So if there is a change in the results format
>> it is just one click, no manual work at all. Maybe we could switch to such
>> an approach?
>>
>> Best,
>>
>> Felix
>>
>>
>>
>>
>> 2012/10/24 Leroy Finn <finnle@tcd.ie>
>>
>>> I agree Dom and Yves that we should discuss it in Lyon so that everyone
>>> knows. Also I have one or two others to add to that discussion in Lyon
>>> before i reproduce the output :)
>>>
>>> Thanks,
>>> Leroy
>>>
>>>
>>> On 24 October 2012 14:13, Dominic Jones <Dominic.Jones@scss.tcd.ie>wrote:
>>>
>>>> Cool, will add this for discussion during the test-suite session.
>>>>
>>>> Dom
>>>>
>>>>
>>>> --
>>>> Dominic Jones | Research Assistant
>>>> KDEG, Trinity College Dublin, Ireland.
>>>>
>>>>
>>>>
>>>>
>>>> On 24 Oct 2012, at 14:04, Yves Savourel <ysavourel@enlaso.com> wrote:
>>>>
>>>> > Hi Dom,
>>>> >
>>>> > Sure. While we work out the issues, there is probably no need for the
>>>> Web site to be always up-to-date. A link to Github would make clear to any
>>>> potential implementers where to get the files.
>>>> >
>>>> > There are probably ways to update the Web site automatically, or at
>>>> least partially automatically. If only the content of a file changes, one
>>>> should be able to just update it from Github. We do that with batch files
>>>> in our development tree. Anyways, maybe something to talk about in Lyon.
>>>> >
>>>> > Cheers,
>>>> > -yves
>>>> >
>>>> > -----Original Message-----
>>>> > From: Dominic Jones [mailto:Dominic.Jones@scss.tcd.ie]
>>>> > Sent: Wednesday, October 24, 2012 6:54 AM
>>>> > To: Yves Savourel
>>>> > Cc: 'Multilingual Web LT-TESTS Public'
>>>> > Subject: Re: Test suite specs
>>>> >
>>>> > Hi Yves.
>>>> >
>>>> > There is an additional problem in that every time we make a change to
>>>> the output or input files we spend a great deal of time updating the
>>>> website with new links. We're dealing with 100+ files now… So I'd like to
>>>> discuss the below during Lyon next week, and agree on the exact format they
>>>> / we would like to see, so that we're not spending an excess of time
>>>> re-generating output files of their file names as the target moves. One
>>>> solution to this is that we just use github for the collection of input and
>>>> output files, making the website redundant, until we're sure that the
>>>> creases have been iron out in exactly how we want the output to be formed.
>>>> Its much easier to just update GitHub than it is to update the website.
>>>> >
>>>> > Are you happy to spend some time next week discussing this, leaving
>>>> this thread open until then?
>>>> >
>>>> > Hope that makes sense.
>>>> >
>>>> > Dom
>>>> > --
>>>> > Dominic Jones | Research Assistant
>>>> > KDEG, Trinity College Dublin, Ireland.
>>>> >
>>>> >
>>>> >
>>>> >
>>>> > On 24 Oct 2012, at 12:47, Yves Savourel <ysavourel@enlaso.com> wrote:
>>>> >
>>>> >> Hi Leroy, all,
>>>> >>
>>>> >> I would tend to disagree :)
>>>> >>
>>>> >> Those files are test results that are going to be (and are already
>>>> in our case) used in unit tests in the builds of some production-grade
>>>> applications. They simply must be predictable and consistent. Not all
>>>> comparison tool can deal with different ordering of the lines for example,
>>>> or present/absent white spaces.
>>>> >>
>>>> >> The only way to have a predictable line order is to sort the
>>>> attributes alphabetically. So I think we should do that.
>>>> >>
>>>> >> For the trailing whitespace, it’s doesn’t matter if they are there
>>>> or not, but they need to be either always there or never.
>>>> >>
>>>> >> We could update the files to reflect that, but it is more efficient
>>>> in the long run to make sure the process you are using to generate them
>>>> does it by itself.
>>>> >>
>>>> >> Cheers,
>>>> >> -yves
>>>> >>
>>>> >>
>>>> >> From: Leroy Finn [mailto:finnle@tcd.ie]
>>>> >> Sent: Wednesday, October 24, 2012 3:08 AM
>>>> >> To: Fredrik Liden
>>>> >> Cc: Multilingual Web LT-TESTS Public
>>>> >> Subject: Re: Test suite specs
>>>> >>
>>>> >> The alphabetical order is not a major problem for my comparison
>>>> engine once the output lines in the actual files are correct.  You can
>>>> include them if you want its no big deal if you don't want the trailing tab
>>>> and empty line at the end. The main thing is that the output lines are
>>>> correct and the ordering is not that important.
>>>> >>
>>>> >> Thanks,
>>>> >> Leroy
>>>> >>
>>>> >> On 23 October 2012 21:36, Fredrik Liden <fliden@enlaso.com> wrote:
>>>> >> Hi Leroy,
>>>> >>
>>>> >> Thanks for adding me.
>>>> >>
>>>> >> In example 6 and 7 there is an instance of
>>>> >>
>>>> >> <item type="title" its:translate="yes">
>>>> >>
>>>> >> The test suite result is:
>>>> >> /doc/info[1]/item[1]     its:translate="yes"
>>>> >> /doc/info[1]/item[1]/@type     its:translate="no"
>>>> >> /doc/info[1]/item[1]/@its:translate      its:translate="no"
>>>> >>
>>>> >> Is this correct or should  the attributes in alphabetical order? I
>>>> think there might be instances in some other categories as well.
>>>> >> /doc/info[1]/item[1]     its:translate="yes"
>>>> >> /doc/info[1]/item[1]/@its:translate      its:translate="no"
>>>> >> /doc/info[1]/item[1]/@type     its:translate="no"
>>>> >>
>>>> >> An email from 8/31 that mentions alphabetical order since the xml
>>>> parsers do not guarantee attribute order.
>>>> >>
>>>> >> Btw, I noticed each line in the test files ends with  \t\r\n. (Tab
>>>> and Linebreak),  I wonder if the trailing \t should be there. There’s also
>>>> a trailing empty row at the end of each result file, should we include that?
>>>> >>
>>>> >> Just in case we need to update our current test result engine so the
>>>> file comparison test passes.
>>>> >>
>>>> >> Cheers,
>>>> >> Fredrik
>>>> >>
>>>> >>
>>>> >
>>>> >
>>>> >
>>>>
>>>>
>>>>
>>>>
>>>
>>
>>
>> --
>> Felix Sasaki
>> DFKI / W3C Fellow
>>
>>
>


-- 
Felix Sasaki
DFKI / W3C Fellow