RE: Conversion of MS CSS 2.1 tests to reftests

> From: Simon Fraser [mailto:smfr@me.com]
> Sent: Tuesday, September 28, 2010 4:32 PM
> To: Sylvain Galineau
> Cc: Geoffrey Sneddon; Anne van Kesteren; L. David Baron; John Jansen;
> fantasai; Arron Eicholz; public-css-testsuite@w3.org
> Subject: Re: Conversion of MS CSS 2.1 tests to reftests
> 
> On Sep 28, 2010, at 4:14 PM, Sylvain Galineau wrote:
> 
> >> -----Original Message-----
> >> From: Geoffrey Sneddon [mailto:gsneddon@opera.com]
> >> Sent: Tuesday, September 28, 2010 3:33 AM
> >> To: Sylvain Galineau
> >> Cc: Anne van Kesteren; L. David Baron; John Jansen; fantasai; Arron
> >> Eicholz; public-css-testsuite@w3.org
> >> Subject: Re: Conversion of MS CSS 2.1 tests to reftests
> >>
> >> On 21/09/10 18:36, Sylvain Galineau wrote:
> >>> It will be a sunk cost once it is built and running.
> >>
> >> The reftest runner is a sunk cost for everyone apart from MS. I
> expect
> >> it'd take around 1w full-time to convert half the testsuite to
> reftests.
> >
> > Cool. Are you doing that ? It sounded like you need the help of
> others for
> > that part.
> >
> >> Given five browsers (i.e., the number of browser vendors in the WG)
> it
> >> would, at the currently quoted time, take 15 (working) days to run.
> If
> >> we spend 5 days automating stuff and get it down to 7.5 days to run
> the
> >> testsuite for all vendors, we've made a net gain. And that's just
> when
> >> running the testsuite once, and I doubt we're just going to do it
> once.
> >
> > Sure, but some vendors have already spent time running the test suite
> at
> > this stage (Apple,Google) so it may not be a net gain for the purpose
> of
> > this first IR.
> 
> Apple has not run the test suite in full. I've done about 25% of the
> tests so far.

You said on Sunday that "I've been through about 60% of the HTML4 tests 
in the 20100917 suite, and have some feedback". I guess I was over-optimistic
about what 'going through' meant :)

> 
> Any work do reduce the amount of manual labor will help us, even at
> this late stage.
> When we start working on actually fixing issues revealed by the suite,
> it will of course
> be invaluable.

Received on Tuesday, 28 September 2010 23:40:17 UTC