test suite for Web IDL v1

I've spent the last week and a bit putting together some tests for Web 
IDL v1, i.e., the version of the spec that is at:

   http://dev.w3.org/2006/webapi/WebIDL/v1.html

I have the tests in a fork of the web-platform-tests repository, here:

 
https://github.com/heycam/web-platform-tests/tree/submission/heycam/WebIDL-tests-1/WebIDL/tests/submissions/heycam

I've got about 120 tests in there at the moment.  I'll work on getting 
them merged in to the main repository.

What we had discussed before was to base the tests on other 
specifications using Web IDL, and to use two examples of each Web IDL 
feature.  I selected, where I could, APIs that had more than one browser 
implementation.  I also tried to choose the two examples of each Web IDL 
feature from different specifications, though that wasn't always 
possible.  There are also some features, like [NamedPropertiesObject], 
that are used in only one specification.

With all of the features in Web IDL, including the different types, I 
ended up with a fair number of APIs to base the tests on.  You can see 
them sorted by name here:

 
https://github.com/heycam/web-platform-tests/blob/submission/heycam/WebIDL-tests-1/WebIDL/tests/submissions/heycam/features.txt

and sorted by kind of feature here:

 
https://github.com/heycam/web-platform-tests/blob/submission/heycam/WebIDL-tests-1/WebIDL/tests/submissions/heycam/features-by-type.txt

(You might wonder why there are so many different IDL attributes tested, 
and that's because they're being tested for different things.  For 
example, in attribute-001.html and attribute-002.html, I'm testing 
Document.title and HTMLSelectElement.length having a property on the 
interface prototype object with the correct [[Enumerable]] value, etc. 
But in long-001.html and long-002.html, I'm testing HTMLPreElement.width 
and HTMLElement.tabIndex specifically for their handling of JS -> IDL 
long conversions when assigning to them.)

I've listed deficiencies in the test suite here:

 
https://github.com/heycam/web-platform-tests/blob/submission/heycam/WebIDL-tests-1/WebIDL/tests/submissions/heycam/notes.txt

That includes cases of there being only a single use of a given feature, 
things that are used in specs but aren't implemented yet, and two 
features that I couldn't see being used at all.  If you're aware of APIs 
that use these features that I didn't find, please let me know or add 
some tests.

It would be good to get feedback on the selection of APIs I ended up 
with, particularly if people think there are some that should be 
avoided.  Review of the tests themselves and areas of test coverage you 
think I've missed is also welcome of course.  I'm also hoping these 
tests will bring to light bugs or other undesirable requirements of the 
spec.

There are still a number of open issues on the spec which I will get to, 
and I'll be updating the test suite as I go along.

Thanks,

Cameron

Received on Monday, 13 May 2013 04:39:25 UTC