Re: Using the Web Audio API in Web Workers

Thanks, Chris -- Done.

On Wed, Apr 8, 2015 at 4:57 PM, Chris Wilson <cwilso@google.com> wrote:

> These notes would be good to have in the Worker issue on the spec.  Could
> you paste them into https://github.com/WebAudio/web-audio-api/issues/16?
>
> On Wed, Apr 8, 2015 at 1:54 PM, Joe Berkovitz <joe@noteflight.com> wrote:
>
>> Hi Ashley,
>>
>> What I can tell you is that it's definitely come up a lot, and was even
>> discussed on the most recent conference call since we were already talking
>> about the need for WebMIDI access in workers.
>>
>> Your implementation ideas are also helpful. Right now we are weighing the
>> priorities of the things that will make the biggest immediate difference in
>> the utility of the spec. Having this input makes it easier to understand
>> how audio in Web Workers might be done, and who it would affect.
>>
>> Best
>> ...Joe
>>
>> On Wed, Apr 8, 2015 at 12:11 PM, Ashley Gullen <ashley@scirra.com> wrote:
>>
>>> We develop a HTML5 game engine and our goal is to ultimately have the
>>> entire engine running from a Web Worker, off the main thread.
>>>
>>> There is progress on getting canvas rendering from workers specced. The
>>> last major hurdle to running an entire game engine in a worker is audio
>>> playback. Currently AFAIK Web Audio is only specced to work from the main
>>> thread, and it is a relatively complex API, so if a game engine is running
>>> in a worker there would need to be a bridge between the worker and the main
>>> thread with lots of postMessage going on, which could impact performance
>>> and add latency while adding complexity. Having Web Audio available in Web
>>> Workers would solve this. Has this been investigated yet?
>>>
>>> Ideally AudioContext could be made available in WorkerGlobalScope.
>>> However I can immediately see a number of issues:
>>>
>>> - integration with Media Elements would not be available (e.g.
>>> createMediaElementSource), because they are part of the DOM which is not
>>> available in a Worker
>>> - integration with Media Streams would not be available, including from
>>> getUserMedia, since these are not available in a Worker (AFAIK)
>>> - selection of audio input and output devices is currently only
>>> available on the main thread (although this may not be a problem if device
>>> ID strings can be posted to the worker)
>>> - ideally types like AudioBuffer would be Transferrable
>>>
>>> Implementation ideas:
>>> - a WorkerAudioContext with the unsupported features removed (e.g.
>>> anything using media elements or media streams) but otherwise working
>>> identically to AudioContext
>>> - a WorkerAudioContext with all features, but some way of integrating
>>> with media elements/media streams from the main thread (e.g. by ID
>>> strings/object URLs)
>>> - a WorkerAudioContext with some kind of specialised WorkerMediaStream
>>> that can access an object URL created from the main thread and route audio
>>> from either a media element or media stream to the WorkerAudioContext
>>> - a WorkerAudioContext with all features, including speccing Media
>>> Streams for workers! (possibly very difficult)
>>> - a Transferrable AudioContextProxy that can be obtained from an
>>> existing AudioContext on the main thread and sent to a worker. Calls to
>>> AudioContextProxy would simply be forwarded to the existing AudioContext
>>> similar to what a postMessage implementation would do, so it's really using
>>> the same context. AudioContextProxy would also be missing media
>>> element/media stream features, but these could still be implemented from
>>> the main thread since it can use all features there. This is probably the
>>> simplest solution and significantly reduces the burden of bridging the APIs
>>> in JS (reducing the bridge to only the features necessary to cover media
>>> elements/media streams), but in the long term implementing MediaStreams in
>>> workers seems like it would be desirable too.
>>>
>>> Any thoughts?
>>>
>>> Ashley Gullen
>>> Scirra.com
>>>
>>>
>>
>>
>> --
>> .            .       .    .  . ...Joe
>>
>> *Joe Berkovitz*
>> President
>>
>> *Noteflight LLC*
>> Boston, Mass.
>> phone: +1 978 314 6271
>> www.noteflight.com
>> "Your music, everywhere"
>>
>
>


-- 
.            .       .    .  . ...Joe

*Joe Berkovitz*
President

*Noteflight LLC*
Boston, Mass.
phone: +1 978 314 6271
www.noteflight.com
"Your music, everywhere"