RE: CSS 2.1 test suite feedback: slicing and dicing

> From: Simon Fraser [mailto:smfr@me.com]
> Sent: Monday, September 27, 2010 2:15 PM
> To: Sylvain Galineau
> Cc: L. David Baron; public-css-testsuite@w3.org
> Subject: Re: CSS 2.1 test suite feedback: slicing and dicing
> 
> On Sep 27, 2010, at 2:03 PM, Sylvain Galineau wrote:
> 
> >> From: L. David Baron [mailto:dbaron@dbaron.org]
> >> Sent: Monday, September 27, 2010 1:48 PM
> >> To: Sylvain Galineau
> >> Cc: Simon Fraser; public-css-testsuite@w3.org
> >> Subject: Re: CSS 2.1 test suite feedback: slicing and dicing
> >>
> >> On Monday 2010-09-27 20:31 +0000, Sylvain Galineau wrote:
> >>> "In particular, the border-color tests test the same color-parsing
> >> issues again and again, for each border side (175 tests per side).
> >>> I don't think this is useful. I think it's find to for the suite to
> >> make some assumptions about how implementations operate, namely
> >>> that the same code will be used to parse colors for each side, so
> >> there is no need to test each individually."
> >>>
> >>> That is reasonable feedback wrt testing the implementability of the
> >> spec. David Baron gave us the same feedback from the beginning.
> >>> I disagree that it is OK from a conformance/interop standpoint to
> >> only test top or left and never check right and bottom.
> >>
> >> Thorough testing of parsing equivalence is much better tested using
> >> script, not individual interactive tests.
> >
> > I'm not sure I understand why it's necessarily better if it allows
> means
> > testing CSSOM/DOM L2 Style. It's certainly easier to author the
> testcase
> > but if we're testing the CSS2.1 spec shouldn't there be a static
> testcase as
> > well ?
> >
> > I will concede that I can see how top/right/bottom/left could/should
> be one
> > testcase i.e. failing one is like failing for all 4. I'm not sure
> it's important
> > enough to make the change at this point though.
> 
> 
> Doing so would eliminate at least 525 tests. Seems worthwhile to me.

Actually, the time to make the change and verifying it would likely be somewhat
larger than what it'd take all browser vendors combined to review them as 3/4 
are very quickly scanned. Given where we are in the timeline, given complaints 
about the number of seconds in three days, the burden of tracking changes to the 
test suite etc., and as this doesn't fix anything and just moves tests around in 
the file system, I'd consider it a nice-to-have. It certainly does not sound like 
something that can block or significantly delay an IR submission.

(And while I think it's reasonable, I do defer to the folks involved who do testing 
for a living...)

Received on Monday, 27 September 2010 21:26:57 UTC