Re: Web Workers

On Wed, Feb 29, 2012 at 3:23 PM, Robert O'Callahan <robert@ocallahan.org>wrote:

> On Wed, Feb 22, 2012 at 7:33 AM, Chris Rogers <crogers@google.com> wrote:
>
>> There is not yet an audio workers implementation in WebKit, but please
>> follow this thread:
>> http://lists.w3.org/Archives/Public/public-audio/2012JanMar/0225.html
>> http://lists.w3.org/Archives/Public/public-audio/2012JanMar/0245.html
>>
>
> Sorry, I did not receive these messages and I don't understand why :-(. I
> blame GMail... Anyway, to answer Dmitry's questions:
>
> Looking at Robert's example (
>> http://people.mozilla.org/~roc/stream-demos/tone.js), the
>> 'onprocessmedia'
>> callback allocates a Float32Array on every invocation. This is wasteful
>> and
>> degrades performance.
>>
>
> Actually you can just rewrite the code like this:
>
> var output;
> self.onprocessmedia = function (event) {
>   var len = event.audioLength;
>   if (!output || output.length < len) {
>     output = new Float32Array(len);
>   }
>   ...
>   event.writeAudio(output);
> }
>
> The writeAudio method still does a copy. Maybe that copy can be avoided
> with an alternative design using transfers, but simply preallocating an
> output buffer as Dmitry suggested isn't quite the right solution since an
> onprocessmedia handler can output any amount of audio, including none.
> Instead we probably should have writeAudio take ownership of the buffer and
> neuter it, or alternatively keep the current copy semantics for writeAudio
> and have a separate transferAudio method that uses the transfer semantics.
>

So these two proposals handle transfer-out but not transfer-in. As you say,
it will either be a copy in a writeAudio, or, if we have a transferAudio
method, an allocation on every callback (because you cannot reuse a
transferred-out array buffer).
If event has a perallocated output buffer, we will avoid any copies and
allocations.


>
> Both current proposals suggest that any worker can handle only one
>> JavaScriptAudioNode/ProcessedMediaStream (since the pocessing nodes run a
>> predefined callback on the worker).
>>
>
> This is definitely not the case for ProcessedMediaStream. There is no
> problem with using the same Worker for several ProcessedMediaStreams. If
> you want to have different types of processing performed in the same
> Worker, you could use a field of the 'params' object in each onprocessmedia
> event to distinguish them.
>

so you would have something like:
self.onprocessmediaevent = function(event) {
   if (event,param.routine == "sinGenerator") { ... }
   else if (event.param.routine == "fobarEffect") { ... }
   ....
}

so you would basically have a dispatch in JavaScript?
This has several drawback:
1) dispatch in JavaScript takes precious cycles that might have been used
processing audio
2) methods structured like this are harder to optimize by JavaScript engine
JIT
3) this requires param to have a value and be transferred to worker on
every call - this takes valuable cylces.
4) the code looks ugly :)

My proposal, that lets you specify a callback name, solves all of that.
Moreover, I can see other benefits:
a. no need to add extra methods to WorkerContext.idl
b. ease of code use and re-use. The user can package her library of audio
effects, where all audio effects have different names but conform to the
same signature, and then easily import those into worker and    call them -
no need for any boilerplate in the worker code itself.



>
> However, I think we should not encourage authors to multiplex processing
> nodes onto Workers by hand, since the optimal assignment could depend on
> the details of the browser and platform (e.g. how heavyweight Workers are,
> and how many cores are available to the browser).
>

It looks like a simple UA-independent guideline that can be given is this:
- if your audio processing nodes are sequential in the graph, dispatch them
onto the same worker
- if they are parallel, dispatch them on different workers.


>
> In some UAs (at least in all WebKit-based ones), worker is a relatively
>> heavy thing (it is a whole OS thread, as well as associated JS execution
>> context resources), so requiring to spawn too many of them is wasteful.
>>
>
> I recommend making Workers as lightweight as possible using thread pools
> and other such mechanisms. This benefits every application that uses
> Workers.
>

Great recommendation but not easily followed.

>
> Instead of making params a field of event object, maybe it will be cleaner
>> to make params an extra argument to callback, and also pass initial value
>> on node/stream construction?
>>
>
> I don't understand the benefits of this. Having an event callback with a
> single event object parameter is a standard pattern for Web APIs and I
> don't see a need to deviate from it here.
>

This just makes for nicer looking and more understandable code, both on
node creation and in processing routine.

Cheers,
Dmitry


>
> Rob
> --
> “You have heard that it was said, ‘Love your neighbor and hate your
> enemy.’ But I tell you, love your enemies and pray for those who persecute
> you, that you may be children of your Father in heaven. ... If you love
> those who love you, what reward will you get? Are not even the tax
> collectors doing that? And if you greet only your own people, what are you
> doing more than others?" [Matthew 5:43-47]
>
>