Re: Test suite specs

Hey Felix,

Yeah i do have a script that does it to all the files in this manner. If
you and Yves think alphabetical is best then i will change the output
to reflect this but currently i am reading through the new draft and
getting the new test files up on the site. After the Lyon meeting I will
update the output to alphabetical as i am producing new output anyway for
the new files.

Thanks,
Leroy

On 24 October 2012 22:07, Felix Sasaki <fsasaki@w3.org> wrote:

> Hi Leroy, all,
>
> having a batch file to re-produce the output can indeed save a lot of
> time. For ITS 1.0 we used such a setup, based on a "master file"
> http://www.w3.org/International/its/tests/input-file-list.xml
> that generates the HTML overview document
> http://www.w3.org/International/its/tests/
> and the expected results. So if there is a change in the results format it
> is just one click, no manual work at all. Maybe we could switch to such an
> approach?
>
> Best,
>
> Felix
>
>
>
>
> 2012/10/24 Leroy Finn <finnle@tcd.ie>
>
>> I agree Dom and Yves that we should discuss it in Lyon so that everyone
>> knows. Also I have one or two others to add to that discussion in Lyon
>> before i reproduce the output :)
>>
>> Thanks,
>> Leroy
>>
>>
>> On 24 October 2012 14:13, Dominic Jones <Dominic.Jones@scss.tcd.ie>wrote:
>>
>>> Cool, will add this for discussion during the test-suite session.
>>>
>>> Dom
>>>
>>>
>>> --
>>> Dominic Jones | Research Assistant
>>> KDEG, Trinity College Dublin, Ireland.
>>>
>>>
>>>
>>>
>>> On 24 Oct 2012, at 14:04, Yves Savourel <ysavourel@enlaso.com> wrote:
>>>
>>> > Hi Dom,
>>> >
>>> > Sure. While we work out the issues, there is probably no need for the
>>> Web site to be always up-to-date. A link to Github would make clear to any
>>> potential implementers where to get the files.
>>> >
>>> > There are probably ways to update the Web site automatically, or at
>>> least partially automatically. If only the content of a file changes, one
>>> should be able to just update it from Github. We do that with batch files
>>> in our development tree. Anyways, maybe something to talk about in Lyon.
>>> >
>>> > Cheers,
>>> > -yves
>>> >
>>> > -----Original Message-----
>>> > From: Dominic Jones [mailto:Dominic.Jones@scss.tcd.ie]
>>> > Sent: Wednesday, October 24, 2012 6:54 AM
>>> > To: Yves Savourel
>>> > Cc: 'Multilingual Web LT-TESTS Public'
>>> > Subject: Re: Test suite specs
>>> >
>>> > Hi Yves.
>>> >
>>> > There is an additional problem in that every time we make a change to
>>> the output or input files we spend a great deal of time updating the
>>> website with new links. We're dealing with 100+ files now… So I'd like to
>>> discuss the below during Lyon next week, and agree on the exact format they
>>> / we would like to see, so that we're not spending an excess of time
>>> re-generating output files of their file names as the target moves. One
>>> solution to this is that we just use github for the collection of input and
>>> output files, making the website redundant, until we're sure that the
>>> creases have been iron out in exactly how we want the output to be formed.
>>> Its much easier to just update GitHub than it is to update the website.
>>> >
>>> > Are you happy to spend some time next week discussing this, leaving
>>> this thread open until then?
>>> >
>>> > Hope that makes sense.
>>> >
>>> > Dom
>>> > --
>>> > Dominic Jones | Research Assistant
>>> > KDEG, Trinity College Dublin, Ireland.
>>> >
>>> >
>>> >
>>> >
>>> > On 24 Oct 2012, at 12:47, Yves Savourel <ysavourel@enlaso.com> wrote:
>>> >
>>> >> Hi Leroy, all,
>>> >>
>>> >> I would tend to disagree :)
>>> >>
>>> >> Those files are test results that are going to be (and are already in
>>> our case) used in unit tests in the builds of some production-grade
>>> applications. They simply must be predictable and consistent. Not all
>>> comparison tool can deal with different ordering of the lines for example,
>>> or present/absent white spaces.
>>> >>
>>> >> The only way to have a predictable line order is to sort the
>>> attributes alphabetically. So I think we should do that.
>>> >>
>>> >> For the trailing whitespace, it’s doesn’t matter if they are there or
>>> not, but they need to be either always there or never.
>>> >>
>>> >> We could update the files to reflect that, but it is more efficient
>>> in the long run to make sure the process you are using to generate them
>>> does it by itself.
>>> >>
>>> >> Cheers,
>>> >> -yves
>>> >>
>>> >>
>>> >> From: Leroy Finn [mailto:finnle@tcd.ie]
>>> >> Sent: Wednesday, October 24, 2012 3:08 AM
>>> >> To: Fredrik Liden
>>> >> Cc: Multilingual Web LT-TESTS Public
>>> >> Subject: Re: Test suite specs
>>> >>
>>> >> The alphabetical order is not a major problem for my comparison
>>> engine once the output lines in the actual files are correct.  You can
>>> include them if you want its no big deal if you don't want the trailing tab
>>> and empty line at the end. The main thing is that the output lines are
>>> correct and the ordering is not that important.
>>> >>
>>> >> Thanks,
>>> >> Leroy
>>> >>
>>> >> On 23 October 2012 21:36, Fredrik Liden <fliden@enlaso.com> wrote:
>>> >> Hi Leroy,
>>> >>
>>> >> Thanks for adding me.
>>> >>
>>> >> In example 6 and 7 there is an instance of
>>> >>
>>> >> <item type="title" its:translate="yes">
>>> >>
>>> >> The test suite result is:
>>> >> /doc/info[1]/item[1]     its:translate="yes"
>>> >> /doc/info[1]/item[1]/@type     its:translate="no"
>>> >> /doc/info[1]/item[1]/@its:translate      its:translate="no"
>>> >>
>>> >> Is this correct or should  the attributes in alphabetical order? I
>>> think there might be instances in some other categories as well.
>>> >> /doc/info[1]/item[1]     its:translate="yes"
>>> >> /doc/info[1]/item[1]/@its:translate      its:translate="no"
>>> >> /doc/info[1]/item[1]/@type     its:translate="no"
>>> >>
>>> >> An email from 8/31 that mentions alphabetical order since the xml
>>> parsers do not guarantee attribute order.
>>> >>
>>> >> Btw, I noticed each line in the test files ends with  \t\r\n. (Tab
>>> and Linebreak),  I wonder if the trailing \t should be there. There’s also
>>> a trailing empty row at the end of each result file, should we include that?
>>> >>
>>> >> Just in case we need to update our current test result engine so the
>>> file comparison test passes.
>>> >>
>>> >> Cheers,
>>> >> Fredrik
>>> >>
>>> >>
>>> >
>>> >
>>> >
>>>
>>>
>>>
>>>
>>
>
>
> --
> Felix Sasaki
> DFKI / W3C Fellow
>
>