Re: [widgets interface] Tests generated from WebIDL

Hi Dom,
Dominique Hazael-Massieux wrote:
> Hi Marcos,
>
> I saw that the test suite for TWI was discussed on the WebApps call
> today:
> http://www.w3.org/2009/11/12-wam-minutes.html#item05
>
> Since the discussion didn’t allude at all to my mail below about
> generated test cases, I thought I would point you to it explicitly in
> case you had missed it.

We had not missed it, but we are not up to the point where we can use 
automated tests. For the purpose of standardization, the creation of 
*manual* test cases forces the Working Group (in this case, me!) to 
verify that every testable assertion is correctly written and testable 
against a product that can claim conformance to a specification.

We believe that an important side effect is that, through the *manual* 
creation of test cases, the test suite can reduce the variability of the 
specification. Reducing the variability makes the specification more 
precise, which in turn can make the specification easier to interpret 
and implement (to paraphrase my testing mentor - you ;)).

On the other hand, automated test generation can generate a large number 
of test cases and is less prone to human errors. But, at the same time, 
it cannot test some things that are written in the prose. For example, a 
AU must not fire Storage events when first populating the preferences 
attribute. This is impossible to express in IDL.

Kind regards,
Marcos

Received on Thursday, 12 November 2009 16:36:06 UTC