WebIDL "Products", was Re: CR exit criteria for Web IDL

On Tuesday, March 20, 2012 at 12:40 AM, Cameron McCormack wrote:

> Here are my thoughts on what we should have as the exit criteria for Web  
> IDL. We talked about this at TPAC last year, so I think people are on  
> board with this.
>  
> We should find two instances of each Web IDL "feature" (whatever that  
> means -- perhaps it means "attribute", "operation", etc., or it might be  
> more fine grained such as "operation argument default value" or  
> "operation argument of type unsigned long") in existing Web  
> specifications and use them as the basis for testing the normative  
> requirements on implementations that Web IDL describes. When we have a  
> choice, we should choose "more stable" specifications/definitions. For  
> example, when choosing what to base our IDL const tests on, choosing  
> Node.ELEMENT_NODE would probably be a good choice.

This seems good in the sense that we can either take existing tests (e.g., DOM ones, or even some of the XHR ones). Or even improve existing test suites by adding the checks for WebIDL specific behavior.   
> There will be some features which exist just for legacy purposes (like  
> say [OverrideBuiltins]) which may not have more than one use in  
> specifications. In these cases we won't need to wait for a second usage  
> of that feature.
>  
> We should make a test suite for all of these features, and the exit  
> criteria should be two passing independent implementations of them.

I don't know how the following relates to, or has any bearing on, the exit criteria, but I feel I need to at least mention it (I've been thinking about this for a few days now, and something about the above is not sitting right with me).  

There are at least five types of products I have seen in the wild that make use of WebIDL directly.

1. Specifications (human written): a number of specifications across a number of working groups define interfaces in terms of WebIDL. E.g., DOM4, HTML5, etc.

2. Specifications (semi-generated):  WebIDL is taken as input, and out comes HTML and content. Examples include Respec.js and widlproc. Respec.js is used by specifications in the DAP WG, as well as IndexDB, as well as others. Widlproc has been used by groups outside the W3C (namely Webinos and WAC).

3. Test generators: WebIDL is taken as input, out comes lots of tests. Example include the generator used to create this test suite:  http://dev.w3.org/2006/waf/widgets-api/test-suite/

4. Code generators: WebIDL is taken as input, a code template comes out as output. Example includes:
http://www.chromium.org/nativeclient/sdk/sdk-experimental/c_salt-design/web-idl-and-c-salt

(I know of one other similar system used on a commercial product, but I don't have a link to share)

5. Browsers: browsers base (or will base) their behavior on it.   

Of course, 5 is obviously the most important - and the one you are basing the exit criteria on…. 1-4 may or many not be significant enough to warrant the attention of the WG when considering conformance requirements and exit criteria (depending on what members feel is the priority). However, I have a feeling that WebIDL will play a pretty important role in QA departments of browser makers. I know it's played a very significant role in QA work I've been involved in over the last few years.  

Just my 2c.  

Kind regards,
Marcos  
--  
Marcos Caceres

Received on Friday, 23 March 2012 14:05:55 UTC