RE: Request for review/approval of reflection tests

Can you move these into Hg?  That would be a good start and should help get feedback on your tests.

Also I'd rather see that the vast majority of tests for attributes align to real use cases.
Else we'll end up with a tons of tests for the same bug over and over. 

For example tabIndex seems to test -36 and also seem to test 2147483647
I don't think 2147483647 is a reasonable tabIndex value, even 32767 seems quite an unreasonable value for tabIndex (who is really going to hit tab 32K times?).

If the spec allows a negative tabIndex that should be a spec bug.
Note that I seems that you can set this attribute it doesn't actually work in any browser that I tried (IE, Chrome, Firefox, Opera)

<!doctype html>
<button tabIndex=10>test</button>
<button tabIndex=-11>test</button>
<button tabIndex=-12>test</button>

-Kris

-----Original Message-----
From: public-html-testsuite-request@w3.org [mailto:public-html-testsuite-request@w3.org] On Behalf Of Aryeh Gregor
Sent: Tuesday, February 15, 2011 5:31 PM
To: public-html-testsuite@w3.org
Subject: Request for review/approval of reflection tests

Some time ago, I wrote a suite of tests for IDL attributes that reflect in content attributes:

http://dev.w3.org/html5/spec/common-dom-interfaces.html#reflecting-content-attributes-in-idl-attributes


In the last couple of days, I cleaned up the tests, added a bunch more tests, ported them to testharness.js, and split them into multiple pages.  The version I propose for review is here:

http://aryeh.name/tests-root/tests/submission/AryehGregor/reflection/reflection-onepage.html


There are also multipage versions that test the same content, but take less time per page to run.

These versions of the tests don't test anything that's only in the WHATWG version of the specification, as far as I can tell -- if they do, that's an error.  (The original version of the tests was written based on the WHATWG version of the spec rather than the W3C version, so something might have crept through.)  They do test microdata, since that's also an HTMLWG deliverable and this is the logical place to test it.

The tests are divided into two major parts: script and data.  The data part is mostly split among the various elements-*.js files, and is around 300 lines.  The script part is in reflection.js, original-harness.js, and new-harness.js, and is around 1000 lines.
The script runs through the data and auto-generates both tests and expected results; it works out to somewhere over 60,000 distinct assertions in around 1500 tests.

I'm not quite done with the tests -- there are some more tests I plan to add to make them more exhaustive -- but I don't expect large changes, so it should be suitable for preliminary review.  Even once approved, the tests will have to be updated regularly for every change to element attributes, but that should be no big deal.

This will obviously take a substantial amount of work to properly review, so I welcome suggestions on how to proceed.

Received on Tuesday, 22 February 2011 03:24:28 UTC