Re: WebIDL test suite time

On Wednesday, April 18, 2012 at 6:15 PM, Travis Leithead wrote:

>  
> I think the main effort for the test suite is to identify:
> 1. the testable assertions (the "features") of WebIDL

I guess this really applies to Section 4 in the spec (ECMAScript binding). The rest can't be tested because the spec does not really define WebIDL Parsers as a conformance class (or it's targeted at spec Editors).  
> 2. 1 or 2 representative "host" interfaces that demonstrate the feature

These would (ideally) have to be based on new specs or interfaces that were defined with WebIDL from the start (so not to bias the results), falling back to existing HTML5 or DOM4 interface definitions as needed for legacy stuff.  

We just need to show a 1 to 1 mapping between existing WebIDL fragments and what is in the WebIDL spec.  

> 3. Author the tests themselves
This is where Aryeh's code would come in, IMO (if it gets updated). It already generates the tests for you and tests all the right bits.  

Otherwise, you would need to create new (fake? test only?) interfaces that would only be used for testing.

> I'd like to help with #1 and possibly #2 above. I can't yet commit to #3 without  
> enlisting help :)  
>  
> I will make an effort to draft of the list of testable assertions (#1)
This could be made trivial is the Editor marked up the RFC2119 keywords. For example:

<p>The foo <em class="ct">MUST</em> do something.</p>  
  
So, IMO, this should be extracted directly from the spec… to avoid duplication and risk your list falling out of sync with the spec.  
  
> and send to this thread by the end of May. I'll also see if I can make
> progress on #2 while I'm at it.


#2 also has the same risk of falling out of sync… would be ideal if the WebIDL was directly extracted from the appropriate specs. Otherwise, this won't be very manageable in the long run.   
  
> [1] http://lists.w3.org/Archives/Public/public-script-coord/2012JanMar/0417.html
> [2] http://lists.w3.org/Archives/Public/public-script-coord/2011OctDec/0158.html



--  
Marcos Caceres

Received on Wednesday, 18 April 2012 18:13:25 UTC