Re: Using the Web Audio API in Web Workers

Yes.  Although realistically, the default output (and input) of Web Audio
isn't going in/out from the main thread anyway - it's going in and out of
the audio thread, and if one believes <audio> data is in the main thread we
already have this problem.  :)

On Wed, Apr 8, 2015 at 3:47 PM, Alex Russell <slightlyoff@google.com> wrote:

> It sounds like this would force the question about "what is the default
> output of Web Audio anyway?". As long as it has only been available from
> the main thread, this hasn't been pressing, but now if we make this
> available in workers (and, perhaps as a result, available inside
> sub-workers constructed from Service Workers), we need to understand how to
> select/attach to an output.
>
> On Wed, Apr 8, 2015 at 2:14 PM, Joe Berkovitz <joe@noteflight.com> wrote:
>
>> Thanks, Chris -- Done.
>>
>> On Wed, Apr 8, 2015 at 4:57 PM, Chris Wilson <cwilso@google.com> wrote:
>>
>>> These notes would be good to have in the Worker issue on the spec.
>>> Could you paste them into
>>> https://github.com/WebAudio/web-audio-api/issues/16?
>>>
>>> On Wed, Apr 8, 2015 at 1:54 PM, Joe Berkovitz <joe@noteflight.com>
>>> wrote:
>>>
>>>> Hi Ashley,
>>>>
>>>> What I can tell you is that it's definitely come up a lot, and was even
>>>> discussed on the most recent conference call since we were already talking
>>>> about the need for WebMIDI access in workers.
>>>>
>>>> Your implementation ideas are also helpful. Right now we are weighing
>>>> the priorities of the things that will make the biggest immediate
>>>> difference in the utility of the spec. Having this input makes it easier to
>>>> understand how audio in Web Workers might be done, and who it would affect.
>>>>
>>>> Best
>>>> ...Joe
>>>>
>>>> On Wed, Apr 8, 2015 at 12:11 PM, Ashley Gullen <ashley@scirra.com>
>>>> wrote:
>>>>
>>>>> We develop a HTML5 game engine and our goal is to ultimately have the
>>>>> entire engine running from a Web Worker, off the main thread.
>>>>>
>>>>> There is progress on getting canvas rendering from workers specced.
>>>>> The last major hurdle to running an entire game engine in a worker is audio
>>>>> playback. Currently AFAIK Web Audio is only specced to work from the main
>>>>> thread, and it is a relatively complex API, so if a game engine is running
>>>>> in a worker there would need to be a bridge between the worker and the main
>>>>> thread with lots of postMessage going on, which could impact performance
>>>>> and add latency while adding complexity. Having Web Audio available in Web
>>>>> Workers would solve this. Has this been investigated yet?
>>>>>
>>>>> Ideally AudioContext could be made available in WorkerGlobalScope.
>>>>> However I can immediately see a number of issues:
>>>>>
>>>>> - integration with Media Elements would not be available (e.g.
>>>>> createMediaElementSource), because they are part of the DOM which is not
>>>>> available in a Worker
>>>>> - integration with Media Streams would not be available, including
>>>>> from getUserMedia, since these are not available in a Worker (AFAIK)
>>>>> - selection of audio input and output devices is currently only
>>>>> available on the main thread (although this may not be a problem if device
>>>>> ID strings can be posted to the worker)
>>>>> - ideally types like AudioBuffer would be Transferrable
>>>>>
>>>>> Implementation ideas:
>>>>> - a WorkerAudioContext with the unsupported features removed (e.g.
>>>>> anything using media elements or media streams) but otherwise working
>>>>> identically to AudioContext
>>>>> - a WorkerAudioContext with all features, but some way of integrating
>>>>> with media elements/media streams from the main thread (e.g. by ID
>>>>> strings/object URLs)
>>>>> - a WorkerAudioContext with some kind of specialised WorkerMediaStream
>>>>> that can access an object URL created from the main thread and route audio
>>>>> from either a media element or media stream to the WorkerAudioContext
>>>>> - a WorkerAudioContext with all features, including speccing Media
>>>>> Streams for workers! (possibly very difficult)
>>>>> - a Transferrable AudioContextProxy that can be obtained from an
>>>>> existing AudioContext on the main thread and sent to a worker. Calls to
>>>>> AudioContextProxy would simply be forwarded to the existing AudioContext
>>>>> similar to what a postMessage implementation would do, so it's really using
>>>>> the same context. AudioContextProxy would also be missing media
>>>>> element/media stream features, but these could still be implemented from
>>>>> the main thread since it can use all features there. This is probably the
>>>>> simplest solution and significantly reduces the burden of bridging the APIs
>>>>> in JS (reducing the bridge to only the features necessary to cover media
>>>>> elements/media streams), but in the long term implementing MediaStreams in
>>>>> workers seems like it would be desirable too.
>>>>>
>>>>> Any thoughts?
>>>>>
>>>>> Ashley Gullen
>>>>> Scirra.com
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> .            .       .    .  . ...Joe
>>>>
>>>> *Joe Berkovitz*
>>>> President
>>>>
>>>> *Noteflight LLC*
>>>> Boston, Mass.
>>>> phone: +1 978 314 6271
>>>> www.noteflight.com
>>>> "Your music, everywhere"
>>>>
>>>
>>>
>>
>>
>> --
>> .            .       .    .  . ...Joe
>>
>> *Joe Berkovitz*
>> President
>>
>> *Noteflight LLC*
>> Boston, Mass.
>> phone: +1 978 314 6271
>> www.noteflight.com
>> "Your music, everywhere"
>>
>
>