ACTION-220 to Investigate into the "wttjs" tool to transform WebIDL example into testsuite

Here are my comments following my ACTION-220 to Investigate into the 
"wttjs" tool to transform WebIDL example into testsuite

I have investigated into the "wttjs" tool [3]:

1- First phase is to prepare an IDL fragment that represents interfaces 
and exceptions that should be implemented by Web browsers. The IDL 
fragment must conform to the Web IDL specification.

I have checked the API for Media Resource 1.0
http://www.w3.org/TR/2010/WD-mediaont-api-1.0-20100309/
http://dev.w3.org/2008/video/mediaann/mediaont-api-1.0/mediaont-api-1.0.html

For both documents, Validation of WebIDL fragments is currently not 
easy. We should use the following markup :
<pre> that encompasses the WebIDL declaration should use a class="idl" 
This would make it possible for the WebIDL checker [1] to detect WebIDL 
fragments directly in the document itself. (this was requested by Dom 
sometime a ago [2]).


2- Second phase is to prepare a JSON file that contains supplemental 
information on interfaces and exceptions to be tested. The JSON file can 
contain two kinds of information:
a- instantiation instructions and
b- crash data.

a - Instantiation instructions: Since an IDL fragment does not describe 
possible ways to instantiate interfaces or exceptions defined in the 
fragment, it is necessary to tell the script how to create host objects 
and host exception objects implementing those interfaces and 
exceptions.If the instantiation instruction for an interface or 
exception is not provided, then any test case that requires an instance 
of the interface or exception will not be generated.

-->   We currently have not defined in our API, ways to instantiate 
interfaces and how to create objects from the browser perspective.

For example using the WebIDL "Identifier" interface

  interface Identifier: MAObject {
                   attribute DOMString value;
                      attribute DOMString type;
                      };

The associated JSON file that contains instantiation instruction data 
would be as follows:
{
   "Identifier": [
     {
       "id": "identifier-object",
       "code": "code.to.createObject (1);"
     }
     , ...
   ]
   , ...
}

Here are some related issues as discussed in the Device APIs and Policy WG
http://www.w3.org/2009/dap/track/issues/44
http://www.w3.org/2009/dap/track/issues/67

b- Crash data: A test case sometimes crashes a Web browser. Since the 
existence of a crashing test case prevents the use of automated test 
harness, a way to avoid execution of test case that is known to crash a 
certain browser is provided.
-->  We may not need the crash data for browsers.

Conclusion:
Currently the "wttjs" tool is not usable, unless we define in our API 
the associated JSON file that contains instantiation instruction data 
for each WebIDL fragtment.

Talking more with Dom, he also explained that this tool is not quite up 
to date with the latest WebIDL spec. therefore we might meet some issues 
with some interfaces, especially when using arrays.

As a conclusion, I think before using this tool, we should defines 
manually some tests cases. This would probably clear up some 
understanding how to construct these objects and then define 
instantiation instruction data on a more general perspective.

Thierry

[1]. http://www.w3.org/2009/07/webidl-check
[2]. 
http://lists.w3.org/Archives/Public/public-media-annotation/2010Feb/0061.html
[3]. http://suika.fam.cx/www/webidl2tests/readme

Received on Monday, 29 March 2010 14:23:55 UTC