WebIDL test suite time

Cameron has previously proposed his thoughts for WebIDL CR exit criteria [1]. 

As the volunteer for coordinating the testing effort, with WebIDL now in CR, I'd like to 
add my exit criteria thoughts regarding the test suite to what Cameron has already said. 

As for the current status, I am not aware of any official WebIDL test suite 
(please correct me if I'm wrong).
 
There is a test harness written by Aryeh Gregor with some initial results here:
http://test.w3.org/html/tests/submission/AryehGregor/interfaces.html

Aryeh has said [2] that his tool is not yet suitable for use yet. I don't believe this 
tool has been updated recently either (to the latest WebIDL changes). I don't know if 
that will soon change. While I like Aryeh's approach, his focus is more on exhaustively 
testing WebIDL based on a specific spec's usage of it. For example, the above test results 
are for a snapshot of the HTML5 specification--I don't think they are suitable as a generic 
WebIDL test suite.
 
We [Microsoft] have a few tests that cover a small swatch of the prototype inheritance 
(a small subset of what WebIDL specifies) on our testing center:
http://samples.msdn.microsoft.com/ietestcenter/#dominheritance. Again, these don't really 
cover the breadth of what is needed.

I think the main effort for the test suite is to identify:
1. the testable assertions (the "features") of WebIDL
2. 1 or 2 representative "host" interfaces that demonstrate the feature
3. Author the tests themselves
 
I'd like to help with #1 and possibly #2 above. I can't yet commit to #3 without 
enlisting help :) 

I will make an effort to draft of the list of testable assertions (#1)
and send to this thread by the end of May. I'll also see if I can make
progress on #2 while I'm at it.

[1] http://lists.w3.org/Archives/Public/public-script-coord/2012JanMar/0417.html
[2] http://lists.w3.org/Archives/Public/public-script-coord/2011OctDec/0158.html


-Travis

Received on Wednesday, 18 April 2012 17:16:21 UTC