Fwd: Dynamic Lifetime and AudioWorker

Hi All,

I had inadvertently posted a reply to Chris alone, and he gave me
thoughtful feedback that I thought it was worth capturing as part of
this thread.  I've forwarded my email here, and will forward his
response email momentarily. (To note, I am doing so with his
permission).

Thanks,
steven


---------- Forwarded message ----------
From: Steven Yi <stevenyi@gmail.com>
Date: Wed, Feb 4, 2015 at 12:10 AM
Subject: Re: Dynamic Lifetime and AudioWorker
To: Chris Wilson <cwilso@google.com>


>> 1. Will it be possible for an AudioWorker to signal a release,
>> similarly to how the above AudioBufferSourceNode can signal turning
>> off? (An imagined use case is below.)
>
>
> Well, sure.  Set its onaudioprocess to null; if there are no references to
> it anywhere, it will get garbage collected.

I have not seen anything in the spec (I'm using the one at
http://webaudio.github.io/web-audio-api/, dated 06 Jan 2015) regarding
setting onaudioprocess to null and what that means.  Is this specified
in a more recent document?  Does that mean that from within an
onaudioprocess, one can do:

onaudioprocess = function(e) {
   ....
   if(done) {
     this.onaudioprocess = null;
   }
}

?

Also, how does this relate to the terminate() function that's exposed
in the AudioWorkerNode?

BTW: I think in terms of release, I should have said something like
"turning off, resulting in an auto-disconnect".  It seems then that if
one sets onaudioprocess to null, one gets the same auto-disconnection
as is described by ABSN when it's done reading it's buffer?

Perhaps a better question: it seems that when ABSN is done with
playback, it no longer has a "playing" reference on itself, and that
is part of what trigger the auto-disconnection.  If correct, should
AudioWorkers have the same ability to have a "playing" reference?  Or
if that "playing" reference aspect is tied to the presence or absence
of an onaudioprocess, could that be added to the specification?

>> 2. Does this release signal correspond to the "onended" event handler,
>> discussed in 2.9.1?
>
>
> Not at all.  "onended" does not correlate to the release of an
> AudioBufferSourceNode - in fact, it can't, because if you're firing the
> event you have a reference to the node object, and that will prevent it from
> releasing.
>
> Also, what triggers an ABSN onended is reaching the end of the buffer in
> playback; what should trigger the onended of a worker?  If your answer is
> "that's up to the work implementation", then sure - but that just means you
> want to post an ended event from inside your audio worker back to the main
> thread, which you can absolutely do.  (And set onaudioprocess to null, if
> you like.)  In short - you can do this already, you don't need extra code in
> AudioWorker to be allowed to do so.

Again sorry, I should have said something like auto-disconnect here
rather than release as in GC freeing.  My question in 2. was more
concerned with the ability of an AudioWorker to auto-disconnect from
the graph than any memory related issues.  The use of onended as just
a notification for the user and having nothing to do with internal
releasing or auto-disconnect is clear to me now from your reply,
thanks.

>> 4. Also if 1 & 2, should AudioWorker have an onended event handler
>> property? Also, if this is added, it seems like an
>> AudioWorkerContext's onaudioprocess code would need an "ended"
>> property or a setEnded() function (or playing/isPlaying())
>
>
> I do not believe so.  If you want that semantic on your own node type, you
> can absolutely implement it.  But we don't need to put it in the base audio
> worker, and it does not apply to the vast majority of audio worker
> scenarios.

Since I had confused ending with turning off to auto-disconnect, then
I agree this isn't necessary. User can arbitrarily create any events
via postMessage.

>> 5. Regarding Example 4.2, Figure 7, if the Compressor did not have any
>> the Stream Source and Gain 1 nodes, what prevents the Compressor node
>> from also being released when the AudioBufferSourceNode is released?
>> >From reading the spec, it seems like it would get killed off once the
>> last Node that is attached is detached.
>
>
> If the streaming source (and gain node) were not present, it WOULD be
> released.  The streaming source and gain node are what keeps it alive.
> (again, this presumes you've released any JS references to the
> DynamicCompressor; otherwise it would be maintained in case you connected
> something else to it.

Ah okay, so as long as there is JS ref to the DynamicCompressor, it
will not auto-disconnect.  This brings up another question:
auto-disconnection seems to be dependent on if a node is release-able
in the GC sense.  If that is correct, does that mean if one keeps a
ref, that nodes will not auto-disconnect? Say for that example, if the
user did have a JS ref to the to the One-Shot sound, once that buffer
is complete, does that whole chain within the dotted lines remain
connected to the graph and processing of the nodes continue to occur
(until that last JS ref is released)? (I guess in that scenario it
would be up to the user to use the onended event to explicitly call
disconnect()?)

>> Use Case for AudioWorker: A user initiates a "note" via a MIDI
>> keyboard.  An AudioWorker that implements a combined envelope
>> generator + VCA is used to process the output of a set of
>> OscillatorNodes, and these are all added to the main audio graph. When
>> the user depresses the key, a message is posted to the AudioWorker to
>> go into a release stage.  At the end of the release stage, the
>> AudioWorker could fire an end event, cleaning up the set of nodes.
>
>
> You can do this today.  You don't need an end "event" - you just destroy the
> node.

Okay, just to clarify, when you say "destroy the node", does that mean
to initiate that by the AudioWork itself setting onaudioprocess =
null, as mentioned earlier? (Assuming that releases a "playing"
reference on itself).


>> 2. If 1 above is a correct interpretation, will it be possible for an
>> AudioWorker to specify it's own tail-time?
>
>
> As the audio worker manages its own lifetime, yes it can.

Okay, I think this is getting a little hairy. I'm not sure then how
onaudioprocess and "playing" and "tail-time" references here could
work.  Let's say we have an AudioWorker that implements a reverb
that's part of "note" chain of nodes. For this let's assume there's an
ABSN at the head of the list of nodes. There are no JS refs once the
node is connected into the graph.  For the user-defined AudioWorker to
have the same lifecycle as a ConvolverNode, it would need to be able
to explicitly set a tail-time to get a "tail-time" reference, but also
not have any kind of "playing" reference, so that the node can
auto-disconnect once the tail-time is done.

So how would this work? Is there a plan to expose a tailTime
read/write property in AudioWorkerGlobalContext, so that the node can
set that?


> No, that's as expected - you probably in this case have a normal JS
> reference.  If you do NOT have a normal JS reference, then the Panner could
> (and should) be garbage collected, as there is no way it is ever going to
> generate sound in the future (how would you connect anything to it with no
> JS reference?).

Ah that makes sense. I think I didn't have the rules of the different
references and the relationship to auto-disconnect clear in my head
when I made this comment. By that logic though, let's say a cycle of
nodes is created and is connected to the AudioDestination, but all of
the JS refs are cleared.  Does that mean it's possible to have a group
of nodes where there is no way for the user to disconnect and clean
them up without killing the audiocontext?

Received on Wednesday, 4 February 2015 18:08:02 UTC