Re: asynchronous vs synchronous instantiation of AudioWorkerNodes

Actually, the API the way it is has a subtly different problem.  (I've
posted this back to the issue
<https://github.com/WebAudio/web-audio-api/issues/113> as well.)

There are two things that happen as the node is created (keep in mind the
node is created in the main thread):

1) AudioParams on the Node object become available in the main thread  -
e.g. if you were reimplementing GainNode as an AudioWorker, you'd want to
do something like:

    var gain = createMyGainNode();  // under the hood, does
context.createAudioWorker( "gainnode.js" );
    gain.gain.value = 0.5;

2) The node itself actually starts processing audio in the audio thread.
This CANNOT happen synchronously - the audio thread needs to load the
script and get it running, and even communicat.

There was a suggestion in the thread that we should move AudioParam
creation into the audio worker's script (that is, the AudioParams would be
created from inside the worker script instead of from the main thread),
because then you can have a clean encapsulation inside that script;
however, the problem is that it breaks the first scenario.  You cannot, if
the AudioParams are being created from the worker script, have them created
from inside the worker script without having WorkerNode creation from the
main thread be asynchronous (since the worker has to download and fire up
the script, and then get an async cross-thread message back: it's not
acceptable to block the main thread while initialization happens in the
audio thread);  at the very least, the script above would need to be:

    var gain=null;
    createMyGainNode().then( function (gainNode) {
        gain = gainNode;
        gain.gain.value = 0.5;
    } );

Bleah.  And, worse yet, if you follow this mechanism it means you can't
replicate the behavior of native node types - because they instantly have
their AudioParams available.

*Conclusion:*
Keeping in mind your custom code can encapsulate the worker scripts, I
think it's better if we just presume that "component" scripts will be
main-thread scripts that package up (as blob, e.g.) the worker script, and
then encapsulate a factory method or object constructor.

E.g.:

<script src="worker-based-gain.js"></js>  <!-- packages up the worker, and
exposes MyGainNode constructor -->
<script>
    var gain = new MyGainNode( audioContext );
    gain.gain.value = 0.5;
</script>

There's a separate issue about whether the main thread should get
notification (an event, maybe) when the worker starts actually processing
audio or not.  However, the worker script *could* post a message back on
first audioprocess event if it wanted to do this notification, so my
inclination is not to do this as a blanket "init event" fired back to the
main script worker node.

-Chris

On Thu, Nov 6, 2014 at 9:09 AM, Joseph Berkovitz <joe@noteflight.com> wrote:

> Hi group,
>
> There’s an issue noted in the AudioWorker proposal (issue #113) that I
> want to highlight as a discussion point, so some public exchange can happen
> prior to the next revised AudioWorker proposal from Chris. (One was already
> expected in order to deal with dynamic input/output counts.)  The
> discussion so far has been limited to the github issue thread but it feels
> worthy of broader attention.
>
> The issue is this: the createAudioWorker factory method on AudioContext
> takes a script URL. Currently it returns a Node (see
> http://cwilso.github.io/web-audio-api/#widl-AudioContext-createAudioWorker-AudioWorkerNode-DOMString-scriptURL-unsigned-long-numberOfInputChannels-unsigned-long-numberOfOutputChannels
> ).
>
> That script could take an arbitrary amount of time to load, so the
> returned Node would not be functional at the time of its instantiation. It
> also might not be able to expose AudioParams at that time, if such
> parameters are defined by executing the loaded script (as has been
> proposed). In response Chris W suggested that this factory method might
> return a Promise, not the actual Node.
>
> The Promise makes sense given the asynchrony of script loading, but the
> need to employ a Promise here makes it hard to synchronously construct a
> graph or subgraph using an AudioWorker node as part of that graph. This is
> a very big leap away from the way one uses native nodes, working against
> the notion of architectural layering built on top of the AudioWorker
> concept. It imposes both script-loading overhead and the use of callbacks
> on every instantiation of a node using some given script.
>
> This is not necessarily so bad for use cases in which a fairly static
> graph is built once, for the duration of some audio session. But it’s more
> serious w/r/t use cases that incorporate many individual AudioWorker
> instances in small subgraphs built on demand (e.g. game sound effects or
> musical notes in a synth). It makes it hard to factor out a one-time
> asynchronous operation that loads assets (in this case, the script) from
> repeated instantiation of objects (the nodes) that employ those same assets.
>
> Ideas to date (forgive me if I left yours out, and please re-propose it)
> have included:
>
> - Don’t return a Promise. Continue to have createAudioWorker() return an
> AudioWorkerNode, synchronously. This node presumably isn’t ready to
> function yet, but it could be inserted into the graph. Script loading would
> happen later, after which the node would initialize (possibly posting a
> message back to the main thread to signal this).  This is a bit like what
> happens with other nodes that have delayed initialization e.g. HRTF
> spatialization. However it might leave important script-determined details
> like input/output count up in the air until the script has loaded and run.
>
> - Have createAudioWorker() return Promises that yield individual nodes,
> and live with the potential delays and inconvenience to developers. Script
> loading and execution takes place each time a node is needed. Developers
> could preload scripts into local Blobs or data URLs to minimize delays.
>
> - Have createAudioWorker() return a Promise that yields a factory object
> with a method that synchronously creates nodes, with each node having its
> own AudioWorkerGlobalScope based on the already-loaded script.  Script
> *loading* would thus occur once per factory instance, but script
> *execution* would occur once per node instance. (this is my proposal,
> though I have little idea of what any implementation objections would be.)
>
> - Have createAudioWorker() return a Promise that yields a factory object
> containing a live AudioWorkerGlobalScope, and allow further nodes to be
> instantiated which share this same scope. Script loading and execution both
> occur once per factory, because the scope is shared. However this
> programming model does not isolate node instances’ state from each other.
> (This is my understanding of a proposal that Jussi made, I haven’t been
> able to track down the original. Jussi has said that some simple use of
> closures would make the state bookkeeping easy but I don’t think an example
> was provided.)
>
> It would be great to surface some thinking on this via email, so that when
> we do discuss it on a telcon we’ll have already seen what the landscape
> looks like.
>
> Best,
>
> .            .       .    .  . ...Joe
>
> PS  if you want to debate some other aspect of AudioWorkers, it would be
> great to start a new thread instead of using this one.
>
>