Re: Test suite specs

Hi Leroy, all,

having a batch file to re-produce the output can indeed save a lot of time.
For ITS 1.0 we used such a setup, based on a "master file"
http://www.w3.org/International/its/tests/input-file-list.xml
that generates the HTML overview document
http://www.w3.org/International/its/tests/
and the expected results. So if there is a change in the results format it
is just one click, no manual work at all. Maybe we could switch to such an
approach?

Best,

Felix




2012/10/24 Leroy Finn <finnle@tcd.ie>

> I agree Dom and Yves that we should discuss it in Lyon so that everyone
> knows. Also I have one or two others to add to that discussion in Lyon
> before i reproduce the output :)
>
> Thanks,
> Leroy
>
>
> On 24 October 2012 14:13, Dominic Jones <Dominic.Jones@scss.tcd.ie> wrote:
>
>> Cool, will add this for discussion during the test-suite session.
>>
>> Dom
>>
>>
>> --
>> Dominic Jones | Research Assistant
>> KDEG, Trinity College Dublin, Ireland.
>>
>>
>>
>>
>> On 24 Oct 2012, at 14:04, Yves Savourel <ysavourel@enlaso.com> wrote:
>>
>> > Hi Dom,
>> >
>> > Sure. While we work out the issues, there is probably no need for the
>> Web site to be always up-to-date. A link to Github would make clear to any
>> potential implementers where to get the files.
>> >
>> > There are probably ways to update the Web site automatically, or at
>> least partially automatically. If only the content of a file changes, one
>> should be able to just update it from Github. We do that with batch files
>> in our development tree. Anyways, maybe something to talk about in Lyon.
>> >
>> > Cheers,
>> > -yves
>> >
>> > -----Original Message-----
>> > From: Dominic Jones [mailto:Dominic.Jones@scss.tcd.ie]
>> > Sent: Wednesday, October 24, 2012 6:54 AM
>> > To: Yves Savourel
>> > Cc: 'Multilingual Web LT-TESTS Public'
>> > Subject: Re: Test suite specs
>> >
>> > Hi Yves.
>> >
>> > There is an additional problem in that every time we make a change to
>> the output or input files we spend a great deal of time updating the
>> website with new links. We're dealing with 100+ files now… So I'd like to
>> discuss the below during Lyon next week, and agree on the exact format they
>> / we would like to see, so that we're not spending an excess of time
>> re-generating output files of their file names as the target moves. One
>> solution to this is that we just use github for the collection of input and
>> output files, making the website redundant, until we're sure that the
>> creases have been iron out in exactly how we want the output to be formed.
>> Its much easier to just update GitHub than it is to update the website.
>> >
>> > Are you happy to spend some time next week discussing this, leaving
>> this thread open until then?
>> >
>> > Hope that makes sense.
>> >
>> > Dom
>> > --
>> > Dominic Jones | Research Assistant
>> > KDEG, Trinity College Dublin, Ireland.
>> >
>> >
>> >
>> >
>> > On 24 Oct 2012, at 12:47, Yves Savourel <ysavourel@enlaso.com> wrote:
>> >
>> >> Hi Leroy, all,
>> >>
>> >> I would tend to disagree :)
>> >>
>> >> Those files are test results that are going to be (and are already in
>> our case) used in unit tests in the builds of some production-grade
>> applications. They simply must be predictable and consistent. Not all
>> comparison tool can deal with different ordering of the lines for example,
>> or present/absent white spaces.
>> >>
>> >> The only way to have a predictable line order is to sort the
>> attributes alphabetically. So I think we should do that.
>> >>
>> >> For the trailing whitespace, it’s doesn’t matter if they are there or
>> not, but they need to be either always there or never.
>> >>
>> >> We could update the files to reflect that, but it is more efficient in
>> the long run to make sure the process you are using to generate them does
>> it by itself.
>> >>
>> >> Cheers,
>> >> -yves
>> >>
>> >>
>> >> From: Leroy Finn [mailto:finnle@tcd.ie]
>> >> Sent: Wednesday, October 24, 2012 3:08 AM
>> >> To: Fredrik Liden
>> >> Cc: Multilingual Web LT-TESTS Public
>> >> Subject: Re: Test suite specs
>> >>
>> >> The alphabetical order is not a major problem for my comparison engine
>> once the output lines in the actual files are correct.  You can include
>> them if you want its no big deal if you don't want the trailing tab and
>> empty line at the end. The main thing is that the output lines are correct
>> and the ordering is not that important.
>> >>
>> >> Thanks,
>> >> Leroy
>> >>
>> >> On 23 October 2012 21:36, Fredrik Liden <fliden@enlaso.com> wrote:
>> >> Hi Leroy,
>> >>
>> >> Thanks for adding me.
>> >>
>> >> In example 6 and 7 there is an instance of
>> >>
>> >> <item type="title" its:translate="yes">
>> >>
>> >> The test suite result is:
>> >> /doc/info[1]/item[1]     its:translate="yes"
>> >> /doc/info[1]/item[1]/@type     its:translate="no"
>> >> /doc/info[1]/item[1]/@its:translate      its:translate="no"
>> >>
>> >> Is this correct or should  the attributes in alphabetical order? I
>> think there might be instances in some other categories as well.
>> >> /doc/info[1]/item[1]     its:translate="yes"
>> >> /doc/info[1]/item[1]/@its:translate      its:translate="no"
>> >> /doc/info[1]/item[1]/@type     its:translate="no"
>> >>
>> >> An email from 8/31 that mentions alphabetical order since the xml
>> parsers do not guarantee attribute order.
>> >>
>> >> Btw, I noticed each line in the test files ends with  \t\r\n. (Tab and
>> Linebreak),  I wonder if the trailing \t should be there. There’s also a
>> trailing empty row at the end of each result file, should we include that?
>> >>
>> >> Just in case we need to update our current test result engine so the
>> file comparison test passes.
>> >>
>> >> Cheers,
>> >> Fredrik
>> >>
>> >>
>> >
>> >
>> >
>>
>>
>>
>>
>


-- 
Felix Sasaki
DFKI / W3C Fellow