Re: test suite for Web IDL v1

Hi All - FYI, Cameron updated his Web IDL tests and they can be reviewed 
and run directly via the following, respectively:

* <https://github.com/w3c/web-platform-tests/pull/271>

* 
<http://w3c-test.org/web-platform-tests/submissions/271/WebIDL/tests/submissions/heycam/>

Getting review of these tests is important so any help (even if you only 
review a subset of the tests) will be greatly appreciated.

-Thanks, AB


On 5/13/13 12:38 AM, ext Cameron McCormack wrote:
> I've spent the last week and a bit putting together some tests for Web 
> IDL v1, i.e., the version of the spec that is at:
>
>   http://dev.w3.org/2006/webapi/WebIDL/v1.html
>
> I have the tests in a fork of the web-platform-tests repository, here:
>
>
> https://github.com/heycam/web-platform-tests/tree/submission/heycam/WebIDL-tests-1/WebIDL/tests/submissions/heycam 
>
>
> I've got about 120 tests in there at the moment.  I'll work on getting 
> them merged in to the main repository.
>
> What we had discussed before was to base the tests on other 
> specifications using Web IDL, and to use two examples of each Web IDL 
> feature.  I selected, where I could, APIs that had more than one 
> browser implementation.  I also tried to choose the two examples of 
> each Web IDL feature from different specifications, though that wasn't 
> always possible.  There are also some features, like 
> [NamedPropertiesObject], that are used in only one specification.
>
> With all of the features in Web IDL, including the different types, I 
> ended up with a fair number of APIs to base the tests on.  You can see 
> them sorted by name here:
>
>
> https://github.com/heycam/web-platform-tests/blob/submission/heycam/WebIDL-tests-1/WebIDL/tests/submissions/heycam/features.txt 
>
>
> and sorted by kind of feature here:
>
>
> https://github.com/heycam/web-platform-tests/blob/submission/heycam/WebIDL-tests-1/WebIDL/tests/submissions/heycam/features-by-type.txt 
>
>
> (You might wonder why there are so many different IDL attributes 
> tested, and that's because they're being tested for different things.  
> For example, in attribute-001.html and attribute-002.html, I'm testing 
> Document.title and HTMLSelectElement.length having a property on the 
> interface prototype object with the correct [[Enumerable]] value, etc. 
> But in long-001.html and long-002.html, I'm testing 
> HTMLPreElement.width and HTMLElement.tabIndex specifically for their 
> handling of JS -> IDL long conversions when assigning to them.)
>
> I've listed deficiencies in the test suite here:
>
>
> https://github.com/heycam/web-platform-tests/blob/submission/heycam/WebIDL-tests-1/WebIDL/tests/submissions/heycam/notes.txt 
>
>
> That includes cases of there being only a single use of a given 
> feature, things that are used in specs but aren't implemented yet, and 
> two features that I couldn't see being used at all.  If you're aware 
> of APIs that use these features that I didn't find, please let me know 
> or add some tests.
>
> It would be good to get feedback on the selection of APIs I ended up 
> with, particularly if people think there are some that should be 
> avoided.  Review of the tests themselves and areas of test coverage 
> you think I've missed is also welcome of course.  I'm also hoping 
> these tests will bring to light bugs or other undesirable requirements 
> of the spec.
>
> There are still a number of open issues on the spec which I will get 
> to, and I'll be updating the test suite as I go along.
>
> Thanks,
>
> Cameron
>

Received on Tuesday, 13 August 2013 15:04:17 UTC