Re: Starting a test suite for Web Audio

On Tuesday, May 21, 2013 at 12:16 PM, Chris Lowis wrote:

> 
> Tobie Langel writes:
> > > The Web Audio WG (of which I am the co-chair, and coordinator of our
> > > testing effort), have started work on some tests for the Web Audio
> > > API. You can see our work-in-progress at
> > > 
> > > https://github.com/WebAudio/web-platform-tests
> > 
> > Why not include your tests directly within
> > https://github.com/w3c/web-platform-tests like other WGs are doing
> > (e.g. WebApps)?
> 
> I think we are, they're in the master branch:
> 
> https://github.com/WebAudio/web-platform-tests/tree/master
Sure, but why work off a fork? This alienates external contributors in events like TTWF for example, and prevents you from benefiting from the tooling we're building, such as the coverage tool.
> I wanted the default branch on our github page to have some
> webaudio-specific information (especially contribution legalise and our
> "submission" workflow), hence the webaudio-readme branch. Does that make
> sense?

We're aiming for a single, cross-group process and review system here.

If you have special legal requirements or issues with the common workflow, please let us now.

We're still in the process of documenting all of these things, so now's the right time to help and/or make comments.
> > > At the moment we are discussing the best way to test audio rendering,
> > > as there are two approaches that might work. We've ported two tests
> > > from the Webkit/Blink and Mozilla test suites and are discussing the
> > > pros and cons on our list (public-audio@w3.org (mailto:public-audio@w3.org)
> > > (mailto:public-audio@w3.org)). We've also made a start on using
> > > WebIDL for tests.
> > 
> > Are you using idlharness for this?
> 
> Yes, we're using the webidl2 submodule in the resources submodule, for example:
> 
> https://github.com/WebAudio/web-platform-tests/blob/master/webaudio/the-audio-api/the-gainnode-interface/gain-node-idl.html
> 
> > > No pull requests for the main platform tests yet, but I thought I'd
> > > let you know of our status.
> > > 
> > > A quick question: Is there a "test runner" or something to automate the
> > > running of a large number of tests and summarise the results?
> > 
> > Not yet, but this is planned as part of our testing effort. We're at
> > the funding stage of this effort at present, so if this is something
> > your group or employer is interested in helping fund, please reach out
> > to me directly.
> 
> Ok, thank you. There was some discussion at our recent f2f about how the
> different browser teams present (Mozilla, Google and Apple) run their
> tests as part of the build, so it sounds technically feasible. I'll have
> a chat with our members to see whether we have any suggestions.

We've been liaising with browser vendors for a while about this effort and are well passed the planning stage at this point. Happy to sit on a call with you and discuss the plans more thoroughly.

At this stage we're deep into the funding part of this effort, and this is clearly an area where we could use the help of our members. Here, again, happy to sit on a call and describe what we're looking for more thoroughly.

Best,

--tobie

Received on Tuesday, 21 May 2013 10:49:04 UTC