Re: Starting a test suite for Web Audio

On Tuesday, May 21, 2013 at 1:46 PM, Chris Lowis wrote:
> Perhaps I've misunderstood something about the github workflow you are
> proposing? The readme in https://github.com/w3c/web-platform-tests
> suggests working off a fork
> 
> "
> The way to contribute is just as usual:
> 
> - fork this repository (and make sure you're still relatively in sync with it if you forked a while ago);
> "

Yes. However, that's intended for individual contributors, not WGs.
> There's some really tricky complexities around testing audio, as it is
> quite unlike a lot of the other test suites I have seen. At the moment
> our discussions centre around using a ScriptProcesserNode vs an
> offlineAudioContext to allow reference tests. I think providing a bit of
> extra guidance and a single place to have tests reviewed and discussed
> is quite useful. Also, as our spec is a moving target at this stage, we'd
> like to try and keep our tests in lock-step with changes to the spec,
> via our bug tracker. I think having a place (our "fork") for tentative
> submissions where we can discuss them on our list might help with that.

Possibly. The fact you feel like you have to do that indicates an issue with the overall process imho, which we should strive to fix.
> I don't think it precludes us from benefiting from your developments, as
> we'll regularly submit the tests our group is working on as a pull request when
> we've written tests we're happy with. I expect we'll do that frequently
> when we've come up with a sensible testing strategy. We just need a
> "sandbox" at this stage.

That makes sense, though I feel like you should be able to treat pull requests as sandboxes.
> > If you have special legal requirements or issues with the common
> > workflow, please let us now.
> 
> No, we don't. Quite a lot of our members are invited experts who only
> engage with the W3C through the Web Audio group, I wanted to make the
> process clear to them.

Which is why we want to centralize this as much as possible. More on this soon.
> We do have a number of members who are keen
> to get involved with testing Audio specifically, and may need some help
> to know how to contribute. You will still be able to review the tests we
> write when we send a pull request to
> https://github.com/w3c/web-platform-tests, of course.
> > We're still in the process of documenting all of these things, so
> > now's the right time to help and/or make comments.
> 
> Sure - it took me a long time to work it out myself. By creating a fork,
> as suggested by your readme,

I think you're misunderstanding the forking process. It is really aimed at single contributors, certainly not a good thing for groups to do. 
> and adding some extra audio-specific
> information I'm trying to lesson the barrier to entry for our group,

So the audio-specific info might make sense to keep on your wiki. The rest should really go in a single, shared document.
> and
> hopefully end up with more tests as a result :)

More tests is our common goal. :)
> > We've been liaising with browser vendors for a while about this effort
> > and are well passed the planning stage at this point. Happy to sit on
> > a call with you and discuss the plans more thoroughly.
> > 
> > At this stage we're deep into the funding part of this effort, and
> > this is clearly an area where we could use the help of our members.
> > Here, again, happy to sit on a call and describe what we're looking
> > for more thoroughly.
> 
> Thank you! You'd be welcome to join our call, of course. I'll try and
> schedule another testing chat into one of agendas very soon.

LMK. Funding this effort won't happen unless we have serious help from out members. So your help is appreciated here.

--tobie

Received on Tuesday, 21 May 2013 12:51:10 UTC