W3C home > Mailing lists > Public > public-audio@w3.org > July to September 2012

Re: Help needed with a sync-problem

From: Marcus Geelnard <mage@opera.com>
Date: Sat, 04 Aug 2012 20:33:46 +0000
Message-ID: <20120804203346.aml5fsqt1e044g0k@staff.opera.com>
To: Chris Rogers <crogers@google.com>
Cc: Peter van der Noord <peterdunord@gmail.com>, Chris Wilson <cwilso@google.com>, Jussi Kalliokoski <jussi.kalliokoski@gmail.com>, Adam Goode <agoode@google.com>, public-audio@w3.org
Citerar Chris Rogers <crogers@google.com>:

> On Fri, Aug 3, 2012 at 4:51 PM, Marcus Geelnard <mage@opera.com> wrote:
>
>> Chris, if the JS node ran as a worker, what do you think about
>> implementing it in such a way that the JS worker "onaudioprocess" callback
>> was executed directly in the real-time audio thread, using the internal
>> buffer size (128 samples) as a working set, which should avoid the latency
>> problems?
>
>
>> I can see some issues (e.g. GC and first-run JIT overhead), but I think
>> that they are either minimal/not-a-real-problem, or solvable.
>
>
> Hi Marcus, I don't think this would be workable because you really need to
> run the web worker as "its own" thread with a distinct message loop for
> handling events from setTimeout(), etc.  but a real-time audio thread
> cannot be blocked.

Yes, I suppose you're right (well, it'd still be technically possible,  
but sensitive to bad programming as you said). After all, Web Workers  
were designed to perform asynchronous tasks to begin with.

I guess a more ideal solution would be a more light-weight "worker",  
perhaps called "AudioProcessingContext" or similar to avoid confusions  
with workers. It would not have any blocking API:s (typically a pure  
ECMAScript context + any Web Audio API specific interfaces and the DSP  
API). It would also have less overhead than a full blown worker  
(memory, setup time etc), so be more suitable to have one-per-node.

>  I don't agree that the GC issues are minimal,
> and garbage collection has always been a big problem for real-time audio.

Obviously, the GC strategy is important, and it might be desirable to  
tune the GC specifically for audio processing contexts. I still  
consider it reasonable to require developers to write "GC friendly"  
code (all audio processing code I've written in JS so far uses static  
heap allocation - it's not that hard to do), and it should be possible  
to make GC for such designs quite cheap.

> Many people tell me that using web workers is very
> difficult with no communications with the DOM or other JavaScript APIs only
> available in the main thread.  I still do support JavaScriptAudioNode
> processing in web workers, so please don't misunderstand me.  I just don't
> want to limit audio to *only* happen there.

Also, please don't misunderstand me ;)  I'm all for ease-of-use and  
providing an API that discourages bad designs. I also sympathize with  
the wishes to do processing in the main thread, but right now I try to  
focus on what to me seems to be one of the most important building  
blocks of the audio API (i.e. custom processing nodes) and getting  
that right and as optimal as possible.

/Marcus
Received on Saturday, 4 August 2012 20:34:20 GMT

This archive was generated by hypermail 2.2.0+W3C-0.50 : Saturday, 4 August 2012 20:34:20 GMT