Re: [css3-2d-transforms] New WIP CSS 2D Transforms tests

On Thu, Jan 5, 2012 at 3:40 PM, Aryeh Gregor <ayg@aryeh.name> wrote:
> I recently began contracting for Mozilla on spec and test work
> (formerly I contracted for Google).  My first work item is to write a
> suite of tests for CSS 2D Transforms, so we can verify
> interoperability and it can progress along the REC track.  At David
> Baron's recommendation, I'll write tests that are a combination of
> reftests and testharness.js-based tests.  I started with
> testharness.js (i.e., DOM/JavaScript tests) because that's what I'm
> familiar with from my past work.  Here's my initial test suite:
>
> http://hg.csswg.org/test/file/1f15a2a425ba/contributors/aryehgregor/incoming/2d-transforms.html
>
> (Unfortunately, there's no version that's actually runnable, since the
> raw file option prompts for a download instead of displaying the file
> raw as at dvcs.w3.org.)

I now have a considerably more complete version, which I think is
enough JS tests for now for 2D transforms (reftests are still needed,
and maybe more JS tests later):

http://hg.csswg.org/test/file/4aa327aaf29b/contributors/aryehgregor/incoming/2d-transforms.html

If you don't want to download the raw file, you can run this straight
in your browser to see how it does, although it might not match the
tip of the hg repo and might not exist indefinitely:

http://aryeh.name/tmp/css-test/contributors/aryehgregor/incoming/2d-transforms.html

Things tested:

* Many different parameters given to each transform function defined
in the specification
* Lists of up to three transforms
* Transforms on up to three nested elements
* Every length unit defined in CSS 2.1
* Every angle unit defined in CSS 3 Values and Units
* Different case variations on every keyword used (lowercase,
uppercase, mixed-case)
* Gibberish keywords
* Transform functions called with too many or too few arguments
* Transform functions called with incorrect units (lengths in
<matrix>, "0" for <angle>)
* Several different ways of positioning the test div, including
floats, absolute and relative positioning, and display: inline-block
and table

I tested both the values returned by getComputedStyle(), and the
actual position of the div as returned by getBoundingClientRect().

Browser interoperability is quite good, but every browser I tested
(IE9, Firefox 12.0a1, Chrome 17 dev, Opera Next 12.00 alpha) had at
least one bug.  Almost every test passes in at least two browsers.  As
far as I can tell after brief review, the only exception is a bunch of
tests exemplified by "Computed value for transform-origin with
transform: none; transform-origin: TOP set via CSSOM":

* Opera Next 12.00 alpha gives "50px 0px" as required by the spec.
* IE9 gives "45px 0px" instead, because it computes the value relative
to padding box instead of border box.  (Rendering is correct, only
getComputedStyle() is wrong.)
* Firefox 12.0a1 gives "50% 0%" instead, returning percentages instead
in this case.  https://bugzilla.mozilla.org/show_bug.cgi?id=715946
* Chrome 17 dev gives "40px 0px" instead, because it computes the
value relative to content box instead of border box.  (Rendering is
correct, only getComputedStyle() is wrong.)

It should be pretty easy for at least one of the three implementations
to fix their bug, which would mean that every test passes in two
independent implementations.  However, there are some issues with the
tests because of possible bugs I found in the spec -- see the FIXMEs
in 2d-transforms.html and transforms.js, all of which have links to
bugs.  Also, not every testable spec requirement is tested yet.

Feedback is highly appreciated.  For now, I've started work on a
similar suite of 3D Transforms tests.

Received on Friday, 13 January 2012 20:22:24 UTC