Re: 28 proposals to improve testcase writing guidelines

* Gérard Talbot wrote:
>Finally, if an IT corporation or a dedicated test author (eg Ian
>Hickson) intends to submit more than 100 testcases, then I think such IT
>corporation or test author should read all relevant documents (including
>the one on my 28 proposals) about creating good testcases. So far, only
>3 entities fit such profile and have submitted over 500 tests in the CSS
>2.1 test suite: Microsoft, Mozilla and Ian Hickson.

The goal of the test suite is to identify implementation differences of
a certain kind, as many of them as possible, and as early as possible.
The ideal test case shows that Firefox does one thing and Webkit another
before Microsoft or Opera or whoever else adds a third implementation to
the mix so the CSS Working Group can make it very clear which behavior,
or behaviors, are correct, while it's cheap to make adjustments. What's
needed beyond that is mainly something that allows you to get a rough
idea about coverage. Anything else, filenames, specification links, you
name it, results in fewer and later tests, as many browser vendors are
not investing all that much in contributing to W3C's CSS test suites,
and more requirements are not likely to make them invest more.

Have a look at <http://shadowregistry.org/js/misc/>. That's a series of
JavaScript functions that return either `true` or `false` and it lists
what various browsers return in addition to what your browser returns.
Most of the test cases there are for DOMParser and XMLSerializer, stuff
that Mozilla originally came up with I believe. If Mozilla had provided
this very web page when they first implemented those APIs, there is no
doubt in my mind that you'd today see so many differences in results,
even though no particular effort was made to do anything beyond showing
how browsers differ (or are otherwise broken). If I could submit tests
for CSS features like that, without all the overhead of learning how to
name test files and whatever, I would have submitted a lot. All I would
have to do to add a new test case to the system above is appending a few
lines of code to <http://shadowregistry.org/js/misc/tests.txt> and that
is already rather inconvenient as I would have to come up with a "title"
for the new test case.
-- 
Björn Höhrmann · mailto:bjoern@hoehrmann.de · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

Received on Friday, 9 December 2011 02:01:45 UTC