Re: "Factory-style" AudioWorker spec proposal

Chris,

A followup thought about your first couple of points which have been nagging at me overnight. Here is a change to my proposal that addresses the issues you raised, but avoids side-by-side worker APIs. I think side-by-side is not a good way to go, it proposes divergent models of thinking about audio workers (and the non-factory approach would not allow global read-only state to be shared by nodes which we’ve previously agreed as important).

This version allows synchronous creation of nodes, but also lets developers elect to wait until a factory is fully loaded before setting audio graph manipulation in motion. A factory is required in both scenarios, unifying the API. If one doesn’t care about script loading overhead, one simply uses the factory right away to make nodes. They won’t work until the script is loaded and, of course, until its node creation handler runs to set up onaudioprocess. But there is no promise involved.

If one wishes to avoid script overhead interfering with node creation, one first waits for the factory to fire an “onloaded” event (one would typically do this in a library initializer that preloads a set of audio worker factories). Then one can create nodes at will, knowing that they are ready to function right away.

In both cases, script loading overhead at most applies to the first node created from a factory, because nodes don’t have their own individual scripts.

partial interface AudioContext {
    AudioWorker createAudioWorker(scriptURL);   // synchronously returns a factory, not a node; factory loads asynchronously
}

interface AudioWorkerNodeConfiguration {
    void setNodeConfiguration(inputs, outputs);   // determines configuration of all subsequently created nodes
    void addParameter(name, default);             // adds AudioParam to all subsequently created nodes
}

// Note that AudioWorker can be told how to configure created nodes immediately by implementing the above interface
interface AudioWorker implements AudioWorkerNodeConfiguration { 
    AudioWorkerNode createAudioWorkerNode();
    attribute EventHandler onload;  // fires after script has loaded and run
}

// For nicer packaging, the AudioWorkerGlobalScope can configure nodes too. This is optional but nice.
AudioWorkerGlobalScope implements AudioWorkerNodeConfiguration;

So one can go this way (if you don’t care when nodes become usable and don’t want the script to do node config):

// main thread
var worker = ctx.createAudioWorker(“bitcrusher.js”);
worker.addParameter(“bits”);  // configure in main thread
var node = worker.createAudioWorkerNode();
node.bits.value = 8;

// bitcrusher.js
onaudionodecreated = function(handle) { … }

Or this way (if you want to preload your worker scripts and can let the scripts do the node configuration):

// main thread
var worker = ctx.createAudioWorker(“bitcrusher.js”);
worker.onload = function() {
  var node = worker.createAudioWorkerNode();
  node.bits.value = 8;
};

// bitcrusher.js
addParameter(“bits”);   // configure in worker script
onaudionodecreated = function(handle) { … }

Specing this fully would require some careful statements about how node creation events and message events are queued to the worker global scope and to the as-yet-nonexistent node handles if the global scope hasn’t initialized yet. But that doesn’t seem too hard.

Thoughts?

.            .       .    .  . ...Joe

Joe Berkovitz
President

Noteflight LLC
Boston, Mass.
phone: +1 978 314 6271
www.noteflight.com
"Your music, everywhere"



On Jan 30, 2015, at 4:58 PM, Joseph Berkovitz <joe@noteflight.com> wrote:

> Chris,
> 
> Thanks for responding so quickly, I’m really happy to have your feedback on this!
> 
> On Jan 29, 2015, at 7:47 PM, Chris Wilson <cwilso@google.com> wrote:
> 
>> Fine work.  A few off-the-cuff comments.
>> I had been expecting we will need to do a side-by-side non-factory audio worker and a factory-based one (i.e. this shouldn't replace the AudioWorker I've been spec'ing - it should go alongside.)  This is mostly because you cannot replicate the current API's behavior with this factory proposal[1] (but also the additional complexity).  This obviously would mean the names could not be shared in quite this way.
>> I like the packaging into a single worker script - always did.  I only didn't do it that way because of [1].
> 
> Yes, I read your earlier remarks on preferring the simpler packaging, but I didn’t notice your referencing [1] in that argument — I thought your original argument was more because of the need to immediately surface the AudioParams surfaced right after the creation of the node (not the creation of the context).  My proposal does address that, but where it falls down is in requiring a promise to complete in between creating the AudioContext and creating nodes from it.  I guess these may feel like the same underlying issue to you.
> 
> Another approach is that one could always use a factory, but let it create nodes (promise-free) even before its script has loaded. In which case, yeah, configuration has to go outside the script. I would easily relent on the configuration-packaging issue if it winds up making these two approaches more similar or even identical.
> 
>> The method name should NOT be "createAudioWorker", because it doesn't create and return an object named "AudioWorkerNode" (pattern we've established across the API).  I would suggest this should be "loadAudioWorker" or, preferably, "loadAudioWorkerFactory".  (particularly because the AudioWorker classes can't be shared, due to my first point.
> Sure, I totally agree (and agonized over this somewhat). I’d rather call the interface itself AudioWorkerFactory, actually.
>> I'm not sure why you changed "bits" to be a hard-defined number in the bitCrusher example, but it should really be a Param - otherwise it's not a particularly real-world.  I think you were just trying to give a Transferable example - but I was confused the "var bitsArray = e.parameters.bits;" part - is this a leftover?  Regardless, "bits" as a Param is a key feature of what a user would want from a bit crusher.
> I agree it’s not very real world. And yes, I forgot to take out the bitsArray assignment from the parameters. I kind of rushed this one :-)
> 
> Maybe I’ll just make bits into a Param and let the VU example carry the weight on Transferables… and read on...
>> I'd rather not have an addition Transferable opportunity unless there's a strong demand; it seems more confusing (we'd have to offer guidance on when to use that Transferable vs postMessage vs AudioParam).
> 
> I think I’ve already changed my mind about that too, and now agree with you.
> 
> Holding off on changes till more comments surface...
> 
>