[css3-2d-transforms] New WIP CSS 2D Transforms tests

I recently began contracting for Mozilla on spec and test work
(formerly I contracted for Google).  My first work item is to write a
suite of tests for CSS 2D Transforms, so we can verify
interoperability and it can progress along the REC track.  At David
Baron's recommendation, I'll write tests that are a combination of
reftests and testharness.js-based tests.  I started with
testharness.js (i.e., DOM/JavaScript tests) because that's what I'm
familiar with from my past work.  Here's my initial test suite:

http://hg.csswg.org/test/file/1f15a2a425ba/contributors/aryehgregor/incoming/2d-transforms.html

(Unfortunately, there's no version that's actually runnable, since the
raw file option prompts for a download instead of displaying the file
raw as at dvcs.w3.org.)

The test is probably in a very different style from all existing CSS
tests.  It's in a similar vein to DOM tests I've written over the last
year, e.g., <http://w3c-test.org/webapps/DOMCore/tests/approved/Range-isPointInRange.html>.
 My approach is to programmatically generate many (in some cases
thousands) of combinations of input, then programmatically determine
the appropriate output and check that.  This means that adding another
test is just a line or two.

For instance, to test various values for transform, I wrote a function
that's called like
    testTransform("scale(3)", 3, 0, 0, 1, 0, 0);
which sets style="transform: scale(3)" on a square div, then computes
where the four corners should be by applying the matrix [3, 0, 0, 1,
0, 0] to the initial four corners, then checks that
getBoundingClientRect() returns the right values, as well as checking
getComputedStyle().  This means I can do things like
    testTransform("translate(" + tx + "px, " + ty + "px)", 1, 0, 0, 1, tx, ty);
for many values of tx and ty with very little effort.

This is clearly not a replacement for reftests, because
getBoundingClientRect() might return different results from actually
displayed, and indeed maybe the client doesn't support JavaScript at
all.  Also, a JavaScript-based test is harder to review than a simple
reftest.  However, for the most important CSS UAs (browsers), I think
this is a valuable way to test many more combinations of features than
would be feasible with reftests alone.  I'd appreciate feedback on
this general approach.

One thing I should note: as far as I'm aware, rounding behavior in CSS
is undefined.  As such, I added an arbitrary tolerance of 1.5 pixels
or percentage points to all numeric comparisons.  This is enough to
cause no rounding-related failures in my tests, except for a very
small number in Opera (which deserves it, since
-o-transform:rotate(180deg) changes the height of a square . . .).
Feedback on the exact tolerance I should use would be appreciated.

I've filed a few bugs against the spec based on my work so far:

https://www.w3.org/Bugs/Public/show_bug.cgi?id=15430
https://www.w3.org/Bugs/Public/show_bug.cgi?id=15431
https://www.w3.org/Bugs/Public/show_bug.cgi?id=15432
https://www.w3.org/Bugs/Public/show_bug.cgi?id=15433

Overall, interoperability seems good so far.  IE9 and Chrome 17 dev
both pass all my tests so far, and Firefox 12.0a1 and Opera Next 12.00
alpha fail only a small number.  I still plan to write a lot more
tests of this type, though -- I have a number of TODOs in the file
indicating what I plan to work on over the next several days.

Received on Thursday, 5 January 2012 20:41:57 UTC