Re: Comments on AudioWorklet examples

.            .       .    .  . ...Joe

Joe Berkovitz
President
Noteflight LLC

+1 978 314 6271

49R Day Street
Somerville MA 02144
USA

"Bring music to life"
www.noteflight.com

On Wed, Jun 1, 2016 at 1:42 PM, Hongchan Choi <hongchan@google.com> wrote:

> It took me a day to think about questions you raised. I think we have an
> important decision to make. Please find my responses inline:
>
>
> Hi Hongchan,
>>
>> Thanks for responding! I will attempt some stunted responses from my
>> phone since I'm traveling today, to keep the discussion going.
>>
>> Based on our previous discussion, I'd expect that there needs to be a
>>> subclass constructor that explicitly copies these values out of the
>>> AudioNodeOptions into the new node's AudioParams, like this:
>>>
>>> class BitcrusherNode extends AudioWorkletNode {
>>>   constructor(ctx, options) {
>>>     super(ctx, 'Bitcrusher', options);
>>>     if (options.hasOwnProperty('bitDepth') {
>>>       this.bitDepth.value = options.bitDepth;
>>>     }
>>>     // ...initialize other params, initial state up front, possibly
>>> utilizing postMessage()...
>>> }
>>>
>>
>> Just to confirm: so this is a node definition, not the processor?
>> Initializing these values from the main thread is a definitely better idea,
>> but the pattern here is quite involved and verbose. I think we can start
>> from this idea and try to simplify the syntax to find a sweet spot. I
>> really need to think about having two definitions for both threads. Very
>> explicit, but that also makes us diverge from Worklet infrastructure which
>> I am a bit afraid of.
>>
>>
>> Sorry, I don't see where this diverges from Worklets -- what am I
>> missing?
>>
>
> Worklet spec does not specify (nor expose) main thread interface. The
> class definition is solely for off-main-thread processing. Now I am
> realizing this is the biggest difference between AudioWorklet and other
> worklet variants. Let's say you can define the node representation as shown
> above, how can the browser match the corresponding processor object? That
> part seems also magical to me. In that case, we might have to introduce a
> new method that runs within global scope to define a `name` and to match
> the processor accordingly. Is that okay?
>

The only magic is in the second argument to the AudioWorkletNode
constructor (DOMString name), that determines which registered Processor is
to be used on the audio side.

The only thing new I've done here is say: it's useful to extend
AudioWorkletNode into a subclass (here, that's BitcrusherNode). The
subclass calls the superclass constructor, which has the same signature it
always had: [Constructor(AudioContext audioContext, DOMString name,
optional AudioNodeOptions options)]. The "name" argument is just filled in
by the subclass, so that the person instantiating the subclass doesn't need
to know it. In my above reworked example, it's 'Bitcrusher', just as it was
in your original example where you call new AudioWorkletNode(ctx,
'Bitcrusher', {...}).


>
> The reason I made this a subclass is because otherwise I did not see how
>> to make use of the custom AudioNodeOptions entries that your example
>> employed  to initialize the bitcrusher AudioParams. The options are
>> constructor args, so unless there is some behavior in AudioWorkletNode that
>> automagically initializes AudioParams with matching names, I don't see how
>> else it can happen.
>>
>
>> If you do envision this init behavior then please make it explicit. Would
>> it apply only to AudioParams? (I assume so because nothing else about a
>> Processor can be declared)
>>
>> Of course one could leave the Params out of the ctor options argument and
>> just initialize them after creating the AudioWorkletNode.
>>
>
> One thing I also want to avoid is over-complicating the definition of
> AudioWorkletNode. Your idea makes perfect sense to me, but I think it is
> too burdensome to developers having to define a node and a processor only
> for the initialization of AudioParams. Beyond the parameter initialization,
> what else can we do? You can define a new method, but it is just a wrapper
> function of postMessage(). I agree that being explicit always win at the
> end, so this is something we have to decide.
>

I want to make way for the group to chime in here. Personally [not as
chair] I think it's best if everything about initialization is explicit.

>
>
>> The only opportunity we have to discover parameter descriptors is before
>>> the success of the promise from the script import for a node type.
>>>
>>   So *maybe* it works like this: after the script is imported for an
>>> AudioWorkletProcessor, the Processor gets instantiated once prior to the
>>> success of the Promise and its paramDescriptors getter is examined; the
>>> result determines the set of parameters exposed by subsequent nodes of that
>>> type. The Processor instance is then thrown away; its only purpose was to
>>> figure out what params exist for this node type.
>>>
>>
>> Can you elaborate on this a bit more? I am not sure if I am following you.
>>
>>
>> Along the lines of my previous response, I was asking myself how the API
>> can know what parameters exist for a Node whose Processor doesn't exist
>> yet. Yet it is the Processor that exposes the param descriptors.
>>
>
>> The only way I see for this to happen is for a Processor to be created as
>> part it's registration, solely for the purpose of interrogating its param
>> descriptors.
>>
>
> I am still not sure about what is being asked, but let me try; you cannot
> know the parameters of a node before its instantiation. Isn't this how a
> regular audio node works? After the construction and when created() method
> gets called, the node reference and AudioParams within will be accessible.
> Or are you asking we implement the new feature for AudioWorkletNode? I can
> see the benefit of figuring out parameters from a node that hasn't been
> created, but I believe it is slightly off-topic.
>

I'm not saying one needs to know the parameters of a Node *before* its
instantiation.

I'm saying one needs to know the parameters of a Node *immediately
following* its instantiation. You can do this with a regular node:

    var g = new GainNode(ctx);
    g.gain.value = 1.5;     // g.gain exists right away!

But with AudioWorkletNodes, we can't take this nice behavior for granted.
Because of the synchronous nature of Node creation (you don't wait for a
Promise), the Processor won't exist yet at the exact moment the Node is
created. We want to be able to do this:

   var bc = new AudioWorkletNode(ctx, 'Bitcrusher');  // or, new
BitcrusherNode(ctx) in my version
   bc.bitDepth.value = 1;   // bc.bitDepth is there right away!

It seems to me that bc.bitDepth can only exist as an AudioParam at that
point if we already know the parameter descriptors declared by the
Processor registered for 'Bitcrusher'. And no one on the main thread side
knows exactly when the AudioWorkletProcessor will be created. That is why I
think this information needs to be made available to the main thread side
somehow, as a byproduct of registering the AudioWorkletProcessor on the
audio side.