RE: CSS 2.1 test suite feedback: slicing and dicing

> -----Original Message-----
> From: public-css-testsuite-request@w3.org [mailto:public-css-testsuite-
> request@w3.org] On Behalf Of Sylvain Galineau
> Sent: Monday, September 27, 2010 2:26 PM
> To: Simon Fraser
> Cc: L. David Baron; public-css-testsuite@w3.org
> Subject: RE: CSS 2.1 test suite feedback: slicing and dicing
> 
> > From: Simon Fraser [mailto:smfr@me.com]
> > Sent: Monday, September 27, 2010 2:15 PM
> > To: Sylvain Galineau
> > Cc: L. David Baron; public-css-testsuite@w3.org
> > Subject: Re: CSS 2.1 test suite feedback: slicing and dicing
> >
> > On Sep 27, 2010, at 2:03 PM, Sylvain Galineau wrote:
> >
> > >> From: L. David Baron [mailto:dbaron@dbaron.org]
> > >> Sent: Monday, September 27, 2010 1:48 PM
> > >> To: Sylvain Galineau
> > >> Cc: Simon Fraser; public-css-testsuite@w3.org
> > >> Subject: Re: CSS 2.1 test suite feedback: slicing and dicing
> > >>
> > >> On Monday 2010-09-27 20:31 +0000, Sylvain Galineau wrote:
> > >>> "In particular, the border-color tests test the same color-parsing
> > >> issues again and again, for each border side (175 tests per side).
> > >>> I don't think this is useful. I think it's find to for the suite
> > >>> to
> > >> make some assumptions about how implementations operate, namely
> > >>> that the same code will be used to parse colors for each side, so
> > >> there is no need to test each individually."
> > >>>
> > >>> That is reasonable feedback wrt testing the implementability of
> > >>> the
> > >> spec. David Baron gave us the same feedback from the beginning.
> > >>> I disagree that it is OK from a conformance/interop standpoint to
> > >> only test top or left and never check right and bottom.
> > >>
> > >> Thorough testing of parsing equivalence is much better tested using
> > >> script, not individual interactive tests.
> > >
> > > I'm not sure I understand why it's necessarily better if it allows
> > means
> > > testing CSSOM/DOM L2 Style. It's certainly easier to author the
> > testcase
> > > but if we're testing the CSS2.1 spec shouldn't there be a static
> > testcase as
> > > well ?
> > >
> > > I will concede that I can see how top/right/bottom/left could/should
> > be one
> > > testcase i.e. failing one is like failing for all 4. I'm not sure
> > it's important
> > > enough to make the change at this point though.
> >
> >
> > Doing so would eliminate at least 525 tests. Seems worthwhile to me.
> 
> Actually, the time to make the change and verifying it would likely be
> somewhat larger than what it'd take all browser vendors combined to review
> them as 3/4 are very quickly scanned. Given where we are in the timeline,
> given complaints about the number of seconds in three days, the burden of
> tracking changes to the test suite etc., and as this doesn't fix anything and
> just moves tests around in the file system, I'd consider it a nice-to-have. It
> certainly does not sound like something that can block or significantly delay
> an IR submission.
> 
> (And while I think it's reasonable, I do defer to the folks involved who do
> testing for a living...)
> 
I agree with Sylvain that making that kind of a change now seems like a "like" and not a "must". Incidentally, you can browse to 

border-bottom-width-35.htm
border-bottom-width-36.htm
border-bottom-width-37.htm
border-bottom-width-47.htm

to see why making assumptions typically equals missing bugs.

David Baron and I spoke ever so briefly about this during the face-to-face, and I definitely think we need to make sure that for CSS3+ we are not duplicating equivalencies - I do not want to test the same thing over and over. But as demonstrated in the four tests above, these are not equivalent, and I'm in favor of erring on the side of coverage rather than speed.

-John
> 
> 
>

Received on Monday, 27 September 2010 22:36:24 UTC