Re: Audio Workers - please review

On Fri, Sep 12, 2014 at 4:13 AM, Joseph Berkovitz <joe@noteflight.com>
wrote:

> @Jussi: In the interest of avoiding opinions about how complicated others
> imagine the code is vs. what you imagine…  could you perhaps provide a code
> sample illustrating the API approach that you are talking about, in
> particular showing how multiple nodes for a worker partition their working
> state? I feel that what needs to be understood by devs is more than just
> “how closures work”: it’s how to usefully employ closures to make the API
> work.
>

A reasonable request, and a great suggestion, so I made two examples:

1) A simple example that shows how to handle a state of a node using a
closure, where you can have multiple instances of the node created on the
same worker: https://gist.github.com/jussi-kalliokoski/8dd140276d2db2084025
2) An example that shows how you might handle states of multiple nodes with
different types, using instanciation, similar to how most of the MVC
frameworks handle component-related state:
https://gist.github.com/jussi-kalliokoski/4ac8d1452fd837cd50b6 In this
example you can see that both sine.js and and square.js sort of register
themselves as node types by adding their own event handler to
onaudionodecreated and detecting if the type matches theirs and then doing
their thing.

As you can see, this is not very different from how you build reusable UI
components. Not by adding their state to the global scope, but by
compartmentalizing it. I think this approach will actually feel more
comfortable and reusable to a lot of developers compared to storing state
on the global scope.


> @Implementors: I would love to see some commentary from those in the know,
> on the inevitability of the importScripts/glitching problem that Jussi is
> raising. In particular, I am not sure that initialization of nodes prior to
> their entering the started state (where importScripts would likely be
> invoked) needs to occur in the audio thread or block it. A node can be
> created and scheduled well before it actually starts to handle
> AudioProcessingEvents, and this is often the case even in an application
> utilizing large numbers of nodes. In the current API design, for example,
> audio processing doesn’t ever block waiting for the main thread to do the
> (admittedly minimal) work of creating the nodes that do the processing.
>
> …Joe
>
>
> On Sep 11, 2014, at 5:56 PM, Jussi Kalliokoski <
> jussi.kalliokoski@gmail.com> wrote:
>
> On Thu, Sep 11, 2014 at 9:45 PM, Joseph Berkovitz <joe@noteflight.com>
> wrote:
>
>>
>> On Sep 11, 2014, at 2:06 PM, Jussi Kalliokoski <
>> jussi.kalliokoski@gmail.com> wrote:
>>
>> On Thu, Sep 11, 2014 at 9:01 PM, Joseph Berkovitz <joe@noteflight.com>
>> wrote:
>>
>>> Jussi,
>>>
>>> I agree the issue of importScripts overhead could be pretty major, if
>>> this overhead is in fact likely to be substantial. I am not knowledgeable
>>> about the extent to which browsers optimize imports of the same scripts in
>>> multiple workers. However it is almost a certainty that nodes will want to
>>> exploit substantial libraries.
>>>
>>> I did not see a convenient way in your proposed API to make it easy for
>>> different nodes based on the same worker to partition their ongoing
>>> computational state (maintained in between onaudioprocess callbacks) from
>>> each other, though. Did I miss something? Doesn’t there need to be a
>>> persistent per-node object that can hold this state?
>>>
>>
>> The state can for example be provided by a closure in the
>> onaudionodecreated event handler, or using a global WeakMap that uses the
>> AudioNodeHandles as keys. This makes sure that the state associated to the
>> node is garbage collected when the node is.
>>
>>
>> I see.  Either of the above would work, but it feels like in this model
>> the developer needs to do some more careful bookkeeping about state rather
>> than just sticking state into the node’s dedicated global scope. That makes
>> the approach feel more complex for developers to consume, relative to
>> Chris’s proposal or the original ScriptProcessorNode.
>>
>
> What I think we'll be looking at as the majority of people writing code
> that runs inside the AudioWorkers will be library authors. As an author of
> more libraries than I care to admit, I'd personally rather learn how
> closures work than get a barrage of bug reports from people using my code,
> complaining that my code is making their applications glitchy (which I get
> at the moment, btw), not being able to really do anything about it except
> blame the API and bugs qualifying as blockers as WONTFIX.
>
> The next biggest group, although likely much smaller, will probably be
> people doing experimentation. Learning. While they're at it, they'll
> probably want to learn the language as well.
>
> In short, I don't think we have any target audience for the AudioWorker's
> that'd have significant trouble understanding how closures work. If we do,
> we should probably specify that group and add it to use cases list and
> decide on how high the priority for catering to them is.
>
>
>>
>> …Joe
>>
>
>
>