Re: Conversion of MS CSS 2.1 tests to reftests

On Sep 28, 2010, at 4:14 PM, Sylvain Galineau wrote:

>> -----Original Message-----
>> From: Geoffrey Sneddon [mailto:gsneddon@opera.com]
>> Sent: Tuesday, September 28, 2010 3:33 AM
>> To: Sylvain Galineau
>> Cc: Anne van Kesteren; L. David Baron; John Jansen; fantasai; Arron
>> Eicholz; public-css-testsuite@w3.org
>> Subject: Re: Conversion of MS CSS 2.1 tests to reftests
>> 
>> On 21/09/10 18:36, Sylvain Galineau wrote:
>>> It will be a sunk cost once it is built and running.
>> 
>> The reftest runner is a sunk cost for everyone apart from MS. I expect
>> it'd take around 1w full-time to convert half the testsuite to reftests.
> 
> Cool. Are you doing that ? It sounded like you need the help of others for
> that part. 
> 
>> Given five browsers (i.e., the number of browser vendors in the WG) it
>> would, at the currently quoted time, take 15 (working) days to run. If
>> we spend 5 days automating stuff and get it down to 7.5 days to run the
>> testsuite for all vendors, we've made a net gain. And that's just when
>> running the testsuite once, and I doubt we're just going to do it once.
> 
> Sure, but some vendors have already spent time running the test suite at
> this stage (Apple,Google) so it may not be a net gain for the purpose of
> this first IR. 

Apple has not run the test suite in full. I've done about 25% of the tests so far.

Any work do reduce the amount of manual labor will help us, even at this late stage.
When we start working on actually fixing issues revealed by the suite, it will of course
be invaluable.

Simon

Received on Tuesday, 28 September 2010 23:32:09 UTC