W3C home > Mailing lists > Public > public-audio@w3.org > January to March 2012

Audio processing with web workers

From: Chris Rogers <crogers@google.com>
Date: Fri, 10 Feb 2012 12:42:47 -0800
Message-ID: <CA+EzO0kCwVRJ4yFRWgH7ew3f6ryVg3uVR+UyOcFsq7cCedwjdg@mail.gmail.com>
To: public-audio@w3.org
Cc: Dmitry Lomov <dslomov@google.com>
>From recent discussions it seems that there's general agreement for both
the need to deliver high-performance native audio processing, plus the
ability to have low-level access to the PCM audio stream for direct
processing in JavaScript.  The Web Audio API currently has both, except
that the JavaScriptAudioNode currently runs the JS code in the main thread.
 Robert's proposal includes the ability to run the JS code in a worker
thread, which I agree can be useful.

>From the perspective of how this would work in the Web Audio API, Jussi has
asked about the possibility of extending the JavaScriptAudioNode to accept
a web worker in the constructor.  I've looked in more detail at Robert's
proposal and his simple example which generates a tone, with pitch
controllable by the mouse:
http://people.mozilla.org/~roc/stream-demos/worker-generation.html
http://people.mozilla.org/~roc/stream-demos/tone.js

Aside from our disagreement about larger aspects of the API, my impression
was that Robert's specific approach to workers seems quite reasonable and
could be adopted in an almost identical way in JavaScriptAudioNode.  I've
since discussed some of the technical aspects with Google's web workers
expert Dmitry Lomov.  And he brought up some interesting ideas and
improvements as well, which I hope we can discuss here.

But first, here's my naive translation of Robert's worker's approach to
JavaScriptAudioNode (which should be refined):

partial interface AudioContext {
    JavaScriptAudioNode createJavaScriptNode(in short bufferSize, in short
numberOfInputChannels, in short numberOfOutputChannels,
        in Worker worker);
};

The code would be slightly different with "self.onprocessmedia" ->
"self.onaudioprocess" to match the current JavaScriptAudioNode naming.
 Also, the "event" would be the same as in the current JavaScriptAudioNode,
but could have Roc's "params".

Cheers,
Chris
Received on Friday, 10 February 2012 20:43:18 GMT

This archive was generated by hypermail 2.2.0+W3C-0.50 : Friday, 10 February 2012 20:43:20 GMT