W3C home > Mailing lists > Public > public-css-testsuite@w3.org > June 2012

Re: UAs passing tests if they don't implement a feature

From: L. David Baron <dbaron@dbaron.org>
Date: Wed, 27 Jun 2012 11:03:38 -0700
To: Bjoern Hoehrmann <derhoermi@gmx.net>
Cc: "GΓ©rard Talbot" <css21testsuite@gtalbot.org>, Aryeh Gregor <ayg@aryeh.name>, CSS-testsuite <public-css-testsuite@w3.org>
Message-ID: <20120627180338.GA4145@crum.dbaron.org>
On Tuesday 2012-06-26 01:15 +0200, Bjoern Hoehrmann wrote:
> Take https://bugzilla.mozilla.org/show_bug.cgi?id=571708 as a simple
> example. That's a perfectly good bug report, you can read half a dozen
> of lines of code and see that they should have no effect on what the bug
> is about, `ctx.translate` and `ctx.fillRect` should not affect what the
> `ctx.fillStyle` property is set to, so setting `ctx.fillStyle` to the
> value that it has already been set to should have no effect. It does,
> though, very obviously so, so there is a bug. I should be able to submit
> this bug "to the W3C" without complaints how it's not a "reftest" or how
> my variable naming conventions do not match some guidelines or about how
> I do not have <link> elements linking the sections in the relevant (non-
> existant) Recommendation or whatever else might be wrong with the bug.

I think there's one point here that you're missing, which is that
the argument about a single bug or test doesn't completely extend to
thousands:  when you're talking about hundreds or thousands (or
hundreds of thousands) of tests, automatability of the tests (either
reftests or testharness.js tests) is important.

Otherwise, however, I agree with your points.

-David

-- 
π„ž   L. David Baron                         http://dbaron.org/   𝄂
𝄒   Mozilla                           http://www.mozilla.org/   𝄂
Received on Wednesday, 27 June 2012 18:06:07 GMT

This archive was generated by hypermail 2.2.0+W3C-0.50 : Wednesday, 27 June 2012 18:06:14 GMT