W3C home > Mailing lists > Public > public-audio@w3.org > January to March 2015

Re: "Factory-style" AudioWorker spec proposal

From: Chris Wilson <cwilso@google.com>
Date: Wed, 4 Feb 2015 10:18:15 -0800
Message-ID: <CAJK2wqX3M-pkPtizO-sTc9mrUtYWW70-Z_Q=-CJXVm46sXK4+Q@mail.gmail.com>
To: Joe Berkovitz <joe@noteflight.com>
Cc: Audio Working Group <public-audio@w3.org>
"objection" is a bit of a loaded term.  I was expressing preference.

I think it's clear that the "best" model would be to have all definition of
the parameters in the worker script for encapsulation. Unfortunately, we
can't mirror the native node behavior that way; but I still want to have
it, and I think that should be the "usual" model for nodes that will be
frequently reused (e.g. a new type of filter).

I'd be happier if the parameter set didn't change from node instance to
node instance, though.  I liked the uniformity of "node configuration in
factory using promise, or node configuration in main thread with no
factory", because I think it lets us do that.

I think the overhead for a factory is a bit much for someone who's just
trying to whip out some basic processing, not create a node that is
expected to be used many times in a single app, and would prefer to avoid
it in those cases.

On Tue, Feb 3, 2015 at 5:55 PM, Joe Berkovitz <joe@noteflight.com> wrote:

> Then how about removing the ability to set the node configuration in the
> worker script from this proposal, and requiring all audioparam and node
> configuration to be in the main thread then (as per your original design)?
>
> This still leaves this proposal with a single worker style api, instead of
> two flavors, and removes the objectionable behavior of prototypes that
> change asynchronously due to script loading. At that point the only need to
> wait for loading is to be assured of quick node setup with no script
> loading overhead -- the original motivation for the factory concept.
>
> I am a little puzzled that you object to two ways of configuring nodes
> (and want to remove the one which is a bit more convenient for developers)
> but still want to preserve two modes of using the api. With the above
> change to meet your objection, anyway, what is the remaining motivation for
> a non-factory approach?
>
> .            .       .    .  . ...Joe
>
> *Joe Berkovitz*
> President
> Noteflight LLC
> +1 978 314 6271
> www.noteflight.com
> "Your music, everywhere."
>
> On Feb 3, 2015, at 8:35 PM, Chris Wilson <cwilso@google.com> wrote:
>
> I think that's going to be problematic, because your prototypes change
> over time - audioparams won't be available until after the script has
> loaded or, as I think you're suggesting, the works-as-native-nodes-do
> implementations would need to run their configuration in the main thread
> anyway.  In short, I think you're just pushing the complexity elsewhere.
> Note that when I say side-by-side, I don't think onaudioprocess would be
> duplicated; just the way you create a worker vs a worker factory.
>
> On Sun, Feb 1, 2015 at 2:20 AM, Joseph Berkovitz <joe@noteflight.com>
> wrote:
>
>> Chris,
>>
>> A followup thought about your first couple of points which have been
>> nagging at me overnight. Here is a change to my proposal that addresses the
>> issues you raised, but avoids side-by-side worker APIs. I think
>> side-by-side is not a good way to go, it proposes divergent models of
>> thinking about audio workers (and the non-factory approach would not allow
>> global read-only state to be shared by nodes which we’ve previously agreed
>> as important).
>>
>> This version allows synchronous creation of nodes, but also lets
>> developers elect to wait until a factory is fully loaded before setting
>> audio graph manipulation in motion. A factory is required in both
>> scenarios, unifying the API. If one doesn’t care about script loading
>> overhead, one simply uses the factory right away to make nodes. They won’t
>> work until the script is loaded and, of course, until its node creation
>> handler runs to set up onaudioprocess. But there is no promise involved.
>>
>> If one wishes to avoid script overhead interfering with node creation,
>> one first waits for the factory to fire an “onloaded” event (one would
>> typically do this in a library initializer that preloads a set of audio
>> worker factories). Then one can create nodes at will, knowing that they are
>> ready to function right away.
>>
>> In both cases, script loading overhead at most applies to the first node
>> created from a factory, because nodes don’t have their own individual
>> scripts.
>>
>> partial interface AudioContext {
>>     AudioWorker createAudioWorker(scriptURL);   // synchronously returns
>> a factory, not a node; factory loads asynchronously
>> }
>>
>> interface AudioWorkerNodeConfiguration {
>>     void setNodeConfiguration(inputs, outputs);   // determines
>> configuration of all subsequently created nodes
>>     void addParameter(name, default);             // adds AudioParam to
>> all subsequently created nodes
>> }
>>
>> // Note that AudioWorker can be told how to configure created nodes
>> immediately by implementing the above interface
>> interface AudioWorker implements AudioWorkerNodeConfiguration {
>>     AudioWorkerNode createAudioWorkerNode();
>>     attribute EventHandler onload;  // fires after script has loaded and
>> run
>> }
>>
>> // For nicer packaging, the AudioWorkerGlobalScope can configure nodes
>> too. This is optional but nice.
>> AudioWorkerGlobalScope implements AudioWorkerNodeConfiguration;
>>
>> So one can go this way (if you don’t care when nodes become usable and
>> don’t want the script to do node config):
>>
>> // main thread
>> var worker = ctx.createAudioWorker(“bitcrusher.js”);
>> worker.addParameter(“bits”);  // configure in main thread
>> var node = worker.createAudioWorkerNode();
>> node.bits.value = 8;
>>
>> // bitcrusher.js
>> onaudionodecreated = function(handle) { … }
>>
>> Or this way (if you want to preload your worker scripts and can let the
>> scripts do the node configuration):
>>
>> // main thread
>> var worker = ctx.createAudioWorker(“bitcrusher.js”);
>> worker.onload = function() {
>>   var node = worker.createAudioWorkerNode();
>>   node.bits.value = 8;
>> };
>>
>> // bitcrusher.js
>> addParameter(“bits”);   // configure in worker script
>> onaudionodecreated = function(handle) { … }
>>
>> Specing this fully would require some careful statements about how node
>> creation events and message events are queued to the worker global scope
>> and to the as-yet-nonexistent node handles if the global scope hasn’t
>> initialized yet. But that doesn’t seem too hard.
>>
>> Thoughts?
>>
>> .            .       .    .  . ...Joe
>>
>> *Joe Berkovitz*
>> President
>>
>> *Noteflight LLC*
>> Boston, Mass.
>> phone: +1 978 314 6271
>> www.noteflight.com
>> "Your music, everywhere"
>>
>>
>>
>> On Jan 30, 2015, at 4:58 PM, Joseph Berkovitz <joe@noteflight.com> wrote:
>>
>> Chris,
>>
>> Thanks for responding so quickly, I’m really happy to have your feedback
>> on this!
>>
>> On Jan 29, 2015, at 7:47 PM, Chris Wilson <cwilso@google.com> wrote:
>>
>> Fine work.  A few off-the-cuff comments.
>>
>>
>>    - I had been expecting we will need to do a side-by-side non-factory
>>    audio worker and a factory-based one (i.e. this shouldn't replace the
>>    AudioWorker I've been spec'ing - it should go alongside.)  This is mostly
>>    because you cannot replicate the current API's behavior with this factory
>>    proposal[1] (but also the additional complexity).  This obviously would
>>    mean the names could not be shared in quite this way.
>>
>>
>>    - I like the packaging into a single worker script - always did.  I
>>    only didn't do it that way because of [1].
>>
>>
>> Yes, I read your earlier remarks on preferring the simpler packaging, but
>> I didn’t notice your referencing [1] in that argument — I thought your
>> original argument was more because of the need to immediately surface the
>> AudioParams surfaced right after the creation of the node (not the creation
>> of the context).  My proposal does address that, but where it falls down is
>> in requiring a promise to complete in between creating the AudioContext and
>> creating nodes from it.  I guess these may feel like the same underlying
>> issue to you.
>>
>> Another approach is that one could always use a factory, but let it
>> create nodes (promise-free) even before its script has loaded. In which
>> case, yeah, configuration has to go outside the script. I would easily
>> relent on the configuration-packaging issue if it winds up making these two
>> approaches more similar or even identical.
>>
>>
>>    - The method name should NOT be "createAudioWorker", because it
>>    doesn't create and return an object named "AudioWorkerNode" (pattern we've
>>    established across the API).  I would suggest this should be
>>    "loadAudioWorker" or, preferably, "loadAudioWorkerFactory".  (particularly
>>    because the AudioWorker classes can't be shared, due to my first point.
>>
>> Sure, I totally agree (and agonized over this somewhat). I’d rather call
>> the interface itself AudioWorkerFactory, actually.
>>
>>
>>    - I'm not sure why you changed "bits" to be a hard-defined number in
>>    the bitCrusher example, but it should really be a Param - otherwise it's
>>    not a particularly real-world.  I think you were just trying to give a
>>    Transferable example - but I was confused the "var bitsArray =
>>    e.parameters.bits;" part - is this a leftover?  Regardless, "bits" as a
>>    Param is a key feature of what a user would want from a bit crusher.
>>
>> I agree it’s not very real world. And yes, I forgot to take out the
>> bitsArray assignment from the parameters. I kind of rushed this one :-)
>>
>> Maybe I’ll just make bits into a Param and let the VU example carry the
>> weight on Transferables… and read on...
>>
>>
>>    - I'd rather not have an addition Transferable opportunity unless
>>    there's a strong demand; it seems more confusing (we'd have to offer
>>    guidance on when to use that Transferable vs postMessage vs AudioParam).
>>
>>
>> I think I’ve already changed my mind about that too, and now agree with
>> you.
>>
>> Holding off on changes till more comments surface...
>>
>>
>>
>>
>
Received on Wednesday, 4 February 2015 18:18:43 UTC

This archive was generated by hypermail 2.3.1 : Wednesday, 4 February 2015 18:18:44 UTC