Re: [web-audio-api] Worker-based ScriptProcessorNode (#113)

On Mon, Aug 11, 2014 at 8:16 AM, Joseph Berkovitz <joe@noteflight.com>
wrote:

>
>    1.
>
>    I presume the "scriptURL" argument would be required to support all
>    URL forms referring to a script? In
>
>    particular, Object URLs? That would be needed to generate scripts on
>    the fly.
>
> This hasn’t been discussed but I also presume/hope this would be OK as it
> is allowed for regular Web Workers.
>

Yes, this would work.

>
>    1.
>
>    How would lifetime of the node be managed in this case? If this is
>    expected to be garbage collected like
>    other nodes, then perhaps the "terminate()" method exposed in the
>    Worker interface should be hidden? If not, then would terminate() be
>    expected to get rid of the node? What would it do?
>
> >Great question, and I wonder the same. I’d like to suggest that the same
node lifetime apply to such nodes as to any other >AudioNodes, based on the
audio processing graph properties.
>
>I think that terminate() perhaps does not make sense in this situation and
should throw an exception.

I don't think we should change the model in that way.  We should make
terminate() kill the worker node, and it should be replaced with silence.

>
>    1.
>
>    When should Object URLs passed to nodes be "revoked"?
>
> I don’t see that Object URLs should be revoked by anything except the
> application code that created them. I guess I don’t understand why there
> would be interaction between Object URLs and Web Audio.
>

 Indeed.  It *could* be revoked once the instantiation is finished (i.e.
the contents is "downloaded"), just like a regular worker.  Although as a
side investigation, I note that Workers don't have an "onload" event, which
is going to make "I'm ready to process now" challenging.

>
>    1.
>
>    While it is a welcome change to do the js audio processing independent
>    of the main thread, it has always
>    seemed excessive to me to dedicate a worker to each node, especially
>    if browsers are expected to run
>    workers in separate threads, and especially for low cpu usage nodes. (
> *) It seems like one shared worker
>    <http://www.w3.org/TR/workers/#shared-workers-introduction> per audio
>    context or per script node type (**) may be better from a system
>    resource perspective
>
>
> The proposal is that each worker will be *run directly* *in the audio
> thread* to do its work. So there will not be thread-related overhead
> related to numbers of workers.
>

Precisely.

>
>    1.
>
>    For a script node to function as a usable *source* node type,
>    instantiation/takedown should be light weight -
>    ex: can be instantiated in predictably short time in response to a key
>    press. Does this design choice limit how
>    light weight it can be?
>    2.
>
>    I presume the spec will detail how this node is expected to work with
>    offline audio contexts. While I don't see
>    any conceptual issues there right away, I just want to pop into view
>    that these are expected to effectively run
>    *synchronously* in the case of offline contexts. Lifetime management
>    will also need to be detailed for this case. (Lifetime management of normal
>    nodes for offline contexts still remains an open issue.)
>
> I agree that these should run synchronously for offline contexts, it is
> the cleanest interpretation. However “synchronously” should refer to
> whatever thread manufactures the output of an offline context, and until we
> clean up the offline audio context spec I don’t know if we can completely
> understand what the consequences of this position are.
>

They WILL run synchronously; that's the beauty here.  It's all in one
thread, and the callbacks are synchronous, so we don't have the same
problem we have today (where the audio thread, while processing
offlineaudiocontexts, would need to continuously block and spin its wheels
until new data comes back from a script processor callback in the main
thread).


>
>    1.
>
>    Lastly, it looks like “how to write an onaudioprocess” won’t change
>    with this proposal. So we’re expected to
>    fill the outputBuffer as usual? Is ownership of this buffer is
>    automatically "transferred" to the audio thread
>    afterwards then?
>
> I believe so; Chris, can you clarify?
>

It's not transferred; it's in the same thread.  It's just passed around.
 In fact, a major benefit is that you CAN reuse the same buffers,
eliminating the need to allocate and deallocate new JS objects for every
onaudioprocess callback; this will decrease GC pressure significantly.

> (**): By "script node type", I mean script nodes that run a common
> onaudioprocess function (ex: compressor). Script nodes that depend on
> common source code can use a shared worker
> <http://www.w3.org/TR/workers/#shared-workers-introduction>.
>
> No, shared workers are different; these would all be dedicted workers.