Re: Reference files in the test suite

We should always strive for the first option.  This what I've been doing in
almost all Gecko test cases that I've written, and it is a lot more
reliable since the conditions for passing the test can all be expressed in
the code.  I would be very hesitant to use wave files produced by any of
the existing applications, because if the test fails on a different
implementation, it would be extremely difficult to figure out which
implementation is the buggy one.

We can explore option 2 if we ever want to test something for which we
cannot write code to generate the expected buffer, however I have no such
examples in mind right now.

Cheers,

--
Ehsan
<http://ehsanakhgari.org/>


On Thu, Jul 18, 2013 at 3:29 AM, Chris Lowis <Chris.Lowis@bbc.co.uk> wrote:

> Hello,
>
> The test of the GainNode in our test suite currently looks like this:
>
>
> https://github.com/w3c/web-platform-tests/blob/master/webaudio/the-audio-api/the-gainnode-interface/gain-node-test.html
>
> It's based on the layout test for the same node in the Webkit[1]. In it we
> compare a generate buffer to a known good reference, stored as a wav file.
>
> At the moment, I've lifted the reference files from the webkit layout
> tests, but, from the point of view of the W3C tests, it isn't particularly
> transparent as to where the reference comes from. There's a few options
>
> - generate a reference using mathematical operations in the setup of the
> test in an ArrayBuffer
> - generate a reference using a third-party software as a wave file
> (perhaps using something like Sox[2]), and have a script to document how it
> is produced
> - keep the wav files from the webkit source and use that as our reference.
>
> I think my preference is for the first option, but I wonder if someone
> familiar with the layout tests in blink/webkit (Chris?) could comment? I'd
> guess the references in those tests are there to catch regressions, and are
> regenerated using the engine itself periodically?
>
> Cheers,
>
> Chris
>
>
> [1]
> https://github.com/WebKit/webkit/blob/master/LayoutTests/webaudio/gain.html
> [2] http://sox.sourceforge.net/
>
>
> -----------------------------
> http://www.bbc.co.uk
> This e-mail (and any attachments) is confidential and
> may contain personal views which are not the views of the BBC unless
> specifically stated.
> If you have received it in
> error, please delete it from your system.
> Do not use, copy or disclose the
> information in any way nor act in reliance on it and notify the sender
> immediately.
> Please note that the BBC monitors e-mails
> sent or received.
> Further communication will signify your consent to
> this.
> -----------------------------
>
>