Re: Dynamic Lifetime and AudioWorker

Just following up with this thread, there was also on further
question/response between Chris and myself to add:

[from Steven]

I do have one further question though regarding tailTime.  It's still
not clear how that scenario plays out to me.  Looking at the Blink
AudioNode code, I see that it silencesOutputs if the inputs are silent
and propagatesSilence returns true (which factors in tailTime).  For
an AudioWorker to achieve that, it would have to also to know if an
input has gone silent to enter into a tailTime-like processing.  It's
not clear then how one checks from JS code in an onaudioprocess if
inputs have gone silent.

Hope that's clearer as to what I'm getting at with tailTime and
AudioWorker.  If there's a way to detect silent input so that the
AudioWorker can do it's own work to finish up processing over time,
perfect. If something more is necessary (whether it's tail-time or
some exposing if something is silent, or something else altogether),
would that be best be added to the new issue you added to github or
another issue?

[from Chris]

It's possible we should have an optimization passed in to the
AudioWorker that tells it it has no inputs or that its input is
silent; and possible we should have the same on output.

On Tue, Feb 3, 2015 at 8:55 PM, Chris Wilson <cwilso@google.com> wrote:
> On Wed, Feb 4, 2015 at 11:13 AM, Steven Yi <stevenyi@gmail.com> wrote:
>>
>> 1. Will it be possible for an AudioWorker to signal a release,
>> similarly to how the above AudioBufferSourceNode can signal turning
>> off? (An imagined use case is below.)
>
>
> Well, sure.  Set its onaudioprocess to null; if there are no references to
> it anywhere, it will get garbage collected.
>>
>>
>> 2. Does this release signal correspond to the "onended" event handler,
>> discussed in 2.9.1?
>
>
> Not at all.  "onended" does not correlate to the release of an
> AudioBufferSourceNode - in fact, it can't, because if you're firing the
> event you have a reference to the node object, and that will prevent it from
> releasing.
>
> Also, what triggers an ABSN onended is reaching the end of the buffer in
> playback; what should trigger the onended of a worker?  If your answer is
> "that's up to the work implementation", then sure - but that just means you
> want to post an ended event from inside your audio worker back to the main
> thread, which you can absolutely do.  (And set onaudioprocess to null, if
> you like.)  In short - you can do this already, you don't need extra code in
> AudioWorker to be allowed to do so.
>
>> 3. If so, from the documentation to AudioBufferSourceNode, could some
>> text be added to clarify when an ended event will be fired?  (perhaps
>> something like "An ended event will fire when either the node is set
>> to not loop and the end of audio buffer data is read, or if stop() is
>> explicitly called.")
>
>
> Well, it currently reads "When the playback of the buffer for an
> AudioBufferSourceNode is finished".  It could be expanded, I suppose; but
> it's not triggered by the release of the object, per se.  The causality is
> "finishing playing triggers onended if applicable, and release of the ABSN
> object if there are no references held to it."  Onended will fire whether
> you have a ref handle to the object or not - it just so happens that if you
> DON'T have any references, once the ABSN has finished playing it can go
> away.
>
>>
>> 4. Also if 1 & 2, should AudioWorker have an onended event handler
>> property? Also, if this is added, it seems like an
>> AudioWorkerContext's onaudioprocess code would need an "ended"
>> property or a setEnded() function (or playing/isPlaying())
>
>
> I do not believe so.  If you want that semantic on your own node type, you
> can absolutely implement it.  But we don't need to put it in the base audio
> worker, and it does not apply to the vast majority of audio worker
> scenarios.
>
>>
>> 5. Regarding Example 4.2, Figure 7, if the Compressor did not have any
>> the Stream Source and Gain 1 nodes, what prevents the Compressor node
>> from also being released when the AudioBufferSourceNode is released?
>> >From reading the spec, it seems like it would get killed off once the
>> last Node that is attached is detached.
>
>
> If the streaming source (and gain node) were not present, it WOULD be
> released.  The streaming source and gain node are what keeps it alive.
> (again, this presumes you've released any JS references to the
> DynamicCompressor; otherwise it would be maintained in case you connected
> something else to it.
>
>>
>> Use Case for AudioWorker: A user initiates a "note" via a MIDI
>> keyboard.  An AudioWorker that implements a combined envelope
>> generator + VCA is used to process the output of a set of
>> OscillatorNodes, and these are all added to the main audio graph. When
>> the user depresses the key, a message is posted to the AudioWorker to
>> go into a release stage.  At the end of the release stage, the
>> AudioWorker could fire an end event, cleaning up the set of nodes.
>
>
> You can do this today.  You don't need an end "event" - you just destroy the
> node.
>
>>
>>
>> #####
>> # In regards to 2.3.3, it mentions that:
>>
>> "A tail-time reference which an AudioNode maintains on itself as long
>> as it has any internal processing state which has not yet been
>> emitted. For example, aConvolverNode has a tail which continues to
>> play even after receiving silent input (think about clapping your
>> hands in a large concert hall and continuing to hear the sound
>> reverberate throughout the hall). Some AudioNodes have this property.
>> Please see details for specific nodes."
>>
>> 1. I went to look at the details for ConvolverNode but there was no
>> mention of the tail-time.  I find the text a little confusing.  My
>> first interpretation was that the tail-time is a feature of some nodes
>> to extend its own processing time past when something like stop() is
>> called, as a scenario like would make sense in terms of lifetime.
>> However, I can't seem to make any conclusive interpretation from the
>> spec alone.  Could I request some clarification on tail-time?
>
>
> This is captured in an issue already -
> https://github.com/WebAudio/web-audio-api/issues/414.
>
>>
>> 2. If 1 above is a correct interpretation, will it be possible for an
>> AudioWorker to specify it's own tail-time?
>
>
> As the audio worker manages its own lifetime, yes it can.
>
>>
>> Use Case: User defines a custom reverb using AudioWorker and wants it
>> to have the same behavior in terms of lifetime as a ConvolverNode.
>>
>> ######
>> # In regards to lifetime, I also noted this text (in 2.3.3, point 4):
>>
>> "A connection reference which occurs if another AudioNode is connected to
>> it."
>>
>> 1. Is this correct? It seems backwards to me, and that a connection
>> reference should be established if a Node is itself connected to
>> another node, and that it is either directly or transitively connected
>> to the AudioDestinationNode of the AudioContext.  Otherwise, you can
>> have a PannerNode with no connections to it, connected to the
>> AudioDestinationNode, and it would have neither a normal, playing,
>> connection, nor tail-time reference.  (To note, the text that follows
>> that point mentions cycles connected to the AudioDestinationNode will
>> be kept alive, but that doesn't seem to apply to this as it's not a
>> cycle).
>
>
> No, that's as expected - you probably in this case have a normal JS
> reference.  If you do NOT have a normal JS reference, then the Panner could
> (and should) be garbage collected, as there is no way it is ever going to
> generate sound in the future (how would you connect anything to it with no
> JS reference?).
>
> -C

Received on Wednesday, 4 February 2015 18:11:08 UTC