Fwd: test suite for Web IDL v1

Forwarding this good news from Cameron re tests for Web IDL v1.

-------- Original Message --------
Subject: 	test suite for Web IDL v1
Resent-Date: 	Mon, 13 May 2013 04:39:29 +0000
Resent-From: 	<public-script-coord@w3.org>
Date: 	Mon, 13 May 2013 14:38:46 +1000
From: 	ext Cameron McCormack <cam@mcc.id.au>
To: 	public-script-coord@w3.org <public-script-coord@w3.org>



I've spent the last week and a bit putting together some tests for Web
IDL v1, i.e., the version of the spec that is at:

   http://dev.w3.org/2006/webapi/WebIDL/v1.html

I have the tests in a fork of the web-platform-tests repository, here:


https://github.com/heycam/web-platform-tests/tree/submission/heycam/WebIDL-tests-1/WebIDL/tests/submissions/heycam

I've got about 120 tests in there at the moment.  I'll work on getting
them merged in to the main repository.

What we had discussed before was to base the tests on other
specifications using Web IDL, and to use two examples of each Web IDL
feature.  I selected, where I could, APIs that had more than one browser
implementation.  I also tried to choose the two examples of each Web IDL
feature from different specifications, though that wasn't always
possible.  There are also some features, like [NamedPropertiesObject],
that are used in only one specification.

With all of the features in Web IDL, including the different types, I
ended up with a fair number of APIs to base the tests on.  You can see
them sorted by name here:


https://github.com/heycam/web-platform-tests/blob/submission/heycam/WebIDL-tests-1/WebIDL/tests/submissions/heycam/features.txt

and sorted by kind of feature here:


https://github.com/heycam/web-platform-tests/blob/submission/heycam/WebIDL-tests-1/WebIDL/tests/submissions/heycam/features-by-type.txt

(You might wonder why there are so many different IDL attributes tested,
and that's because they're being tested for different things.  For
example, in attribute-001.html and attribute-002.html, I'm testing
Document.title and HTMLSelectElement.length having a property on the
interface prototype object with the correct [[Enumerable]] value, etc.
But in long-001.html and long-002.html, I'm testing HTMLPreElement.width
and HTMLElement.tabIndex specifically for their handling of JS -> IDL
long conversions when assigning to them.)

I've listed deficiencies in the test suite here:


https://github.com/heycam/web-platform-tests/blob/submission/heycam/WebIDL-tests-1/WebIDL/tests/submissions/heycam/notes.txt

That includes cases of there being only a single use of a given feature,
things that are used in specs but aren't implemented yet, and two
features that I couldn't see being used at all.  If you're aware of APIs
that use these features that I didn't find, please let me know or add
some tests.

It would be good to get feedback on the selection of APIs I ended up
with, particularly if people think there are some that should be
avoided.  Review of the tests themselves and areas of test coverage you
think I've missed is also welcome of course.  I'm also hoping these
tests will bring to light bugs or other undesirable requirements of the
spec.

There are still a number of open issues on the spec which I will get to,
and I'll be updating the test suite as I go along.

Thanks,

Cameron

Received on Tuesday, 21 May 2013 13:23:12 UTC