Re: Using the Web Audio API in Web Workers

It sounds like this would force the question about "what is the default
output of Web Audio anyway?". As long as it has only been available from
the main thread, this hasn't been pressing, but now if we make this
available in workers (and, perhaps as a result, available inside
sub-workers constructed from Service Workers), we need to understand how to
select/attach to an output.

On Wed, Apr 8, 2015 at 2:14 PM, Joe Berkovitz <joe@noteflight.com> wrote:

> Thanks, Chris -- Done.
>
> On Wed, Apr 8, 2015 at 4:57 PM, Chris Wilson <cwilso@google.com> wrote:
>
>> These notes would be good to have in the Worker issue on the spec.  Could
>> you paste them into https://github.com/WebAudio/web-audio-api/issues/16?
>>
>> On Wed, Apr 8, 2015 at 1:54 PM, Joe Berkovitz <joe@noteflight.com> wrote:
>>
>>> Hi Ashley,
>>>
>>> What I can tell you is that it's definitely come up a lot, and was even
>>> discussed on the most recent conference call since we were already talking
>>> about the need for WebMIDI access in workers.
>>>
>>> Your implementation ideas are also helpful. Right now we are weighing
>>> the priorities of the things that will make the biggest immediate
>>> difference in the utility of the spec. Having this input makes it easier to
>>> understand how audio in Web Workers might be done, and who it would affect.
>>>
>>> Best
>>> ...Joe
>>>
>>> On Wed, Apr 8, 2015 at 12:11 PM, Ashley Gullen <ashley@scirra.com>
>>> wrote:
>>>
>>>> We develop a HTML5 game engine and our goal is to ultimately have the
>>>> entire engine running from a Web Worker, off the main thread.
>>>>
>>>> There is progress on getting canvas rendering from workers specced. The
>>>> last major hurdle to running an entire game engine in a worker is audio
>>>> playback. Currently AFAIK Web Audio is only specced to work from the main
>>>> thread, and it is a relatively complex API, so if a game engine is running
>>>> in a worker there would need to be a bridge between the worker and the main
>>>> thread with lots of postMessage going on, which could impact performance
>>>> and add latency while adding complexity. Having Web Audio available in Web
>>>> Workers would solve this. Has this been investigated yet?
>>>>
>>>> Ideally AudioContext could be made available in WorkerGlobalScope.
>>>> However I can immediately see a number of issues:
>>>>
>>>> - integration with Media Elements would not be available (e.g.
>>>> createMediaElementSource), because they are part of the DOM which is not
>>>> available in a Worker
>>>> - integration with Media Streams would not be available, including from
>>>> getUserMedia, since these are not available in a Worker (AFAIK)
>>>> - selection of audio input and output devices is currently only
>>>> available on the main thread (although this may not be a problem if device
>>>> ID strings can be posted to the worker)
>>>> - ideally types like AudioBuffer would be Transferrable
>>>>
>>>> Implementation ideas:
>>>> - a WorkerAudioContext with the unsupported features removed (e.g.
>>>> anything using media elements or media streams) but otherwise working
>>>> identically to AudioContext
>>>> - a WorkerAudioContext with all features, but some way of integrating
>>>> with media elements/media streams from the main thread (e.g. by ID
>>>> strings/object URLs)
>>>> - a WorkerAudioContext with some kind of specialised WorkerMediaStream
>>>> that can access an object URL created from the main thread and route audio
>>>> from either a media element or media stream to the WorkerAudioContext
>>>> - a WorkerAudioContext with all features, including speccing Media
>>>> Streams for workers! (possibly very difficult)
>>>> - a Transferrable AudioContextProxy that can be obtained from an
>>>> existing AudioContext on the main thread and sent to a worker. Calls to
>>>> AudioContextProxy would simply be forwarded to the existing AudioContext
>>>> similar to what a postMessage implementation would do, so it's really using
>>>> the same context. AudioContextProxy would also be missing media
>>>> element/media stream features, but these could still be implemented from
>>>> the main thread since it can use all features there. This is probably the
>>>> simplest solution and significantly reduces the burden of bridging the APIs
>>>> in JS (reducing the bridge to only the features necessary to cover media
>>>> elements/media streams), but in the long term implementing MediaStreams in
>>>> workers seems like it would be desirable too.
>>>>
>>>> Any thoughts?
>>>>
>>>> Ashley Gullen
>>>> Scirra.com
>>>>
>>>>
>>>
>>>
>>> --
>>> .            .       .    .  . ...Joe
>>>
>>> *Joe Berkovitz*
>>> President
>>>
>>> *Noteflight LLC*
>>> Boston, Mass.
>>> phone: +1 978 314 6271
>>> www.noteflight.com
>>> "Your music, everywhere"
>>>
>>
>>
>
>
> --
> .            .       .    .  . ...Joe
>
> *Joe Berkovitz*
> President
>
> *Noteflight LLC*
> Boston, Mass.
> phone: +1 978 314 6271
> www.noteflight.com
> "Your music, everywhere"
>