Re: Starting a test suite for Web Audio

On 05/21/2013 01:46 PM, Chris Lowis wrote:
>
> Tobie Langel writes:

> There's some really tricky complexities around testing audio, as it is
> quite unlike a lot of the other test suites I have seen. At the moment
> our discussions centre around using a ScriptProcesserNode vs an
> offlineAudioContext to allow reference tests. I think providing a bit of
> extra guidance and a single place to have tests reviewed and discussed
> is quite useful. Also, as our spec is a moving target at this stage, we'd
> like to try and keep our tests in lock-step with changes to the spec,
> via our bug tracker. I think having a place (our "fork") for tentative
> submissions where we can discuss them on our list might help with that.
>
> I don't think it precludes us from benefiting from your developments, as
> we'll regularly submit the tests our group is working on as a pull request when
> we've written tests we're happy with. I expect we'll do that frequently
> when we've come up with a sensible testing strategy. We just need a
> "sandbox" at this stage.
>
>>> I wanted the default branch on our github page to have some
>>> webaudio-specific information (especially contribution legalise and our
>>> "submission" workflow), hence the webaudio-readme branch. Does that make
>>> sense?
>>
>> We're aiming for a single, cross-group process and review system here.
>>
>> If you have special legal requirements or issues with the common
>> workflow, please let us now.
>
> No, we don't. Quite a lot of our members are invited experts who only
> engage with the W3C through the Web Audio group, I wanted to make the
> process clear to them. We do have a number of members who are keen
> to get involved with testing Audio specifically, and may need some help
> to know how to contribute. You will still be able to review the tests we
> write when we send a pull request to
> https://github.com/w3c/web-platform-tests, of course.

To be clear about this, there is nothing that stops you/people in your 
wg submitting PRs to web-platform-tests and having the policy that only 
other WG members should review things in the webaudio folder for now. 
That does have a number of advantages; e.g. your tests (including those 
in unaccepted pull requests) will automatically be mirrored on 
w3c-test.org so that people can run them without a local checkout. There 
is also an instance of the critic code review tool set up for the main 
web-platform-tests repository, and this would allow people in your wg to 
get notified of changes/submissions for webaudio tests alone (using the 
filter system). Using this location also makes it more likely that 
people familiar with the infrastructure will comment on whether you are 
following common idioms with testharness.js or idlharness.js.

On the other hand I don't really object if you iterate the testsuite in 
another location at first and then make a submission later. But it does 
mean that you will have worse tooling and could lead to unnecessary work 
later on.

Received on Tuesday, 21 May 2013 11:56:31 UTC