Re: Linking test suites to specs

Le Mar 25 juin 2013 3:14, Tobie Langel a écrit :

> I'm working on a solution to automate running test suites (including
> reftests) and gathering results using WebDriver.

Tobie,

I am for automation of running test suites thanks to reftests. At the
same time, I want to point out that a wide majority of tests do not have
a reftest. And the tests that have a reftest very often have *_not_*
been designed to be reusing an already created reftest; there are now 7
reftests in
http://test.csswg.org/source/approved/css2.1/src/reference/
which are currently associated with 844 tests. This number could be
easily doubled or tripled just by retrofitting or tweaking some hundreds
of tests.

As the overall mass of tests is growing, I do not think that having "n"
tests associated with "n" reftests (a 1-to-1) is best.

Another thing about creating reftests. Often, I had to tune tests to
avoid fractional values, computed font-size not dividable by 2, by 5 and
by 10 in various situations. It is when creating reftests that you can
best discover tests with imprecisions.

Many tests involving inline-blocks, inline-tables, tables have to be
tuned before creating reftests for these.

Gérard
-- 
Contributions to the CSS 2.1 test suite:
http://www.gtalbot.org/BrowserBugsSection/css21testsuite/

CSS 2.1 Test suite RC6, March 23rd 2011:
http://test.csswg.org/suites/css2.1/20110323/html4/toc.html

CSS 2.1 test suite harness:
http://test.csswg.org/harness/

Contributing to to CSS 2.1 test suite:
http://www.gtalbot.org/BrowserBugsSection/css21testsuite/web-authors-contributions-css21-testsuite.html

Received on Tuesday, 25 June 2013 18:21:48 UTC