W3C home > Mailing lists > Public > public-audio@w3.org > April to June 2015

Re: Using the Web Audio API in Web Workers

From: Joe Berkovitz <joe@noteflight.com>
Date: Wed, 8 Apr 2015 16:54:10 -0400
Message-ID: <CA+ojG-Yt0wAXkjbTuF535+c6g044KW7yP5myh4gSupYDBdcm8A@mail.gmail.com>
To: Ashley Gullen <ashley@scirra.com>
Cc: "public-audio@w3.org" <public-audio@w3.org>
Hi Ashley,

What I can tell you is that it's definitely come up a lot, and was even
discussed on the most recent conference call since we were already talking
about the need for WebMIDI access in workers.

Your implementation ideas are also helpful. Right now we are weighing the
priorities of the things that will make the biggest immediate difference in
the utility of the spec. Having this input makes it easier to understand
how audio in Web Workers might be done, and who it would affect.

Best
...Joe

On Wed, Apr 8, 2015 at 12:11 PM, Ashley Gullen <ashley@scirra.com> wrote:

> We develop a HTML5 game engine and our goal is to ultimately have the
> entire engine running from a Web Worker, off the main thread.
>
> There is progress on getting canvas rendering from workers specced. The
> last major hurdle to running an entire game engine in a worker is audio
> playback. Currently AFAIK Web Audio is only specced to work from the main
> thread, and it is a relatively complex API, so if a game engine is running
> in a worker there would need to be a bridge between the worker and the main
> thread with lots of postMessage going on, which could impact performance
> and add latency while adding complexity. Having Web Audio available in Web
> Workers would solve this. Has this been investigated yet?
>
> Ideally AudioContext could be made available in WorkerGlobalScope. However
> I can immediately see a number of issues:
>
> - integration with Media Elements would not be available (e.g.
> createMediaElementSource), because they are part of the DOM which is not
> available in a Worker
> - integration with Media Streams would not be available, including from
> getUserMedia, since these are not available in a Worker (AFAIK)
> - selection of audio input and output devices is currently only available
> on the main thread (although this may not be a problem if device ID strings
> can be posted to the worker)
> - ideally types like AudioBuffer would be Transferrable
>
> Implementation ideas:
> - a WorkerAudioContext with the unsupported features removed (e.g.
> anything using media elements or media streams) but otherwise working
> identically to AudioContext
> - a WorkerAudioContext with all features, but some way of integrating with
> media elements/media streams from the main thread (e.g. by ID
> strings/object URLs)
> - a WorkerAudioContext with some kind of specialised WorkerMediaStream
> that can access an object URL created from the main thread and route audio
> from either a media element or media stream to the WorkerAudioContext
> - a WorkerAudioContext with all features, including speccing Media Streams
> for workers! (possibly very difficult)
> - a Transferrable AudioContextProxy that can be obtained from an existing
> AudioContext on the main thread and sent to a worker. Calls to
> AudioContextProxy would simply be forwarded to the existing AudioContext
> similar to what a postMessage implementation would do, so it's really using
> the same context. AudioContextProxy would also be missing media
> element/media stream features, but these could still be implemented from
> the main thread since it can use all features there. This is probably the
> simplest solution and significantly reduces the burden of bridging the APIs
> in JS (reducing the bridge to only the features necessary to cover media
> elements/media streams), but in the long term implementing MediaStreams in
> workers seems like it would be desirable too.
>
> Any thoughts?
>
> Ashley Gullen
> Scirra.com
>
>


-- 
.            .       .    .  . ...Joe

*Joe Berkovitz*
President

*Noteflight LLC*
Boston, Mass.
phone: +1 978 314 6271
www.noteflight.com
"Your music, everywhere"
Received on Wednesday, 8 April 2015 20:54:38 UTC

This archive was generated by hypermail 2.3.1 : Wednesday, 8 April 2015 20:54:39 UTC