Re: Testing and Test Driver

Hi, folks-

On 7/19/12 12:32 PM, Raymond Toy wrote:
>
> On Thu, Jul 19, 2012 at 12:17 AM, Marcus Geelnard wrote:
>
>> Generally, I'm positive to early testing and DFT (design for test)
>> methodology.

That's great to hear!  I hope that others feel the same way.


>> However, the current state of the Web Audio API poses a
>> few issues:
>
>> * Much of the signal processing behavior of the audio API is
>> undefined, meaning that the majority of semantic tests are currently
>> impossible to write based on the spec. Here, writing the tests and
>> writing the spec would go hand-in-hand, and can probably only be
>> done successfully by the editor.
>
> Having written some tests, I don't think it needs to be that tightly
> coupled to the editor.  Granted, this will cause some delay in updating
> the spec, but for the areas that are incomplete, I doubt this adds any
> additional time because we all need to come to an agreement on what
> details need to be resolved.

I agree with Ray.  I've written tests for SVG (in some cases also for 
thorny low-level issues) that resulted in changes to that spec, for 
which I was not the editor. It's a dynamic WG relationship.


>> * Generally speaking, writing tests would more often than not
>> require changes/additions to the spec (e.g. turning non-normative
>> wording into normative text), and I don't really see how we can do
>> that efficiently unless the test writers are also spec writers in
>> one way or the other.
>
> Sometime back, I added a small enhancement for biquad nodes.  After
> writing the test for that, I just wrote a patch with appropriate text
> for the enhancement, and some additional text describing the 7 biquad
> types and gave it to Chris Rogers.  He incorporated the changes fairly
> quickly.
>
> I guess what I'm saying is that we will probably spend a lot more time
> deciding on what the tests should actually be testing and whether
> they're valid or not than we will on updating the spec to say what we mean.

I strongly agree with Ray here, too. If you have proposed spec changes, 
and especially a great test to back it up, then you can simply submit 
that for review on the email list, and when we come to consensus on the 
issue, the editor can incorporate the resulting spec change.

It is unfair to everyone, and unworkably inefficient, to have the editor 
be the only one to write tests (though any tests from the editor are 
welcome), or to have everyone writing tests to also be an editor. This 
is a task that the whole WG should be doing, as part of the dialog for 
improving the spec, and we have a clear mechanism for aligning the spec 
and the tests: submitting spec patches.


If there is a suggestion buried in your comment that we might consider 
having more than one editor, I think that is a good point to be raised, 
but with the caveat that we do not want to appoint conflicting 
editors... they would have to work closely and fairly harmoniously 
together, or the work will never get done. If it is more efficient to 
have multiple editors, then that's an interesting conversation to have.

Regards-
-Doug

Received on Thursday, 19 July 2012 19:26:45 UTC