Re: UAs passing tests if they don't implement a feature

On Tuesday 2012-06-26 01:15 +0200, Bjoern Hoehrmann wrote:
> Take https://bugzilla.mozilla.org/show_bug.cgi?id=571708 as a simple
> example. That's a perfectly good bug report, you can read half a dozen
> of lines of code and see that they should have no effect on what the bug
> is about, `ctx.translate` and `ctx.fillRect` should not affect what the
> `ctx.fillStyle` property is set to, so setting `ctx.fillStyle` to the
> value that it has already been set to should have no effect. It does,
> though, very obviously so, so there is a bug. I should be able to submit
> this bug "to the W3C" without complaints how it's not a "reftest" or how
> my variable naming conventions do not match some guidelines or about how
> I do not have <link> elements linking the sections in the relevant (non-
> existant) Recommendation or whatever else might be wrong with the bug.

I think there's one point here that you're missing, which is that
the argument about a single bug or test doesn't completely extend to
thousands:  when you're talking about hundreds or thousands (or
hundreds of thousands) of tests, automatability of the tests (either
reftests or testharness.js tests) is important.

Otherwise, however, I agree with your points.

-David

-- 
π„ž   L. David Baron                         http://dbaron.org/   𝄂
𝄒   Mozilla                           http://www.mozilla.org/   𝄂

Received on Wednesday, 27 June 2012 18:06:07 UTC