Re: ScriptProcessorNode off the main thread (Was: Questioning the current direction of the Web Audio API)

2013-10-25 15:17, Joseph Berkovitz skrev:
> Let me add a few points to what Marcus said.
>
> I think the requirements do not necessarily imply the use of regular 
> web Workers, and that we should be careful not to silently pick up 
> unnecessary overhead or behavioral implication from Workers. At the 
> same time there are a number of strong similarities between Workers 
> and JS-nodes-off-the-main-thread, suggesting that it makes sense to 
> look to Workers to solve this problems.
>
> Here are some of the issues with off-the-shelf workers that I see:
>
> - Shared workers, while not having a per-node overhead problem, seem 
> too global in scope. It feels as though we want different working 
> storage for different nodes, even if they share the same script. 
> Global communication between nodes would probably lead to bugs, not 
> advantages.

Yes, I think that we'd need (and want) isolated environments - at least 
in terms of state, variable scoping etc.

On the other hand, I think that most browsers today strive to treat each 
worker as a separate OS thread (though it's not mandated by the spec), 
and it's quite questionable if the notion of one-OS-thread-per-worker 
would be a benefit for audio nodes. Maybe we want to batch node 
processing of several nodes into a single thread (I envision a "pool" of 
audio nodes that need to be executed in some order, and there may be one 
or only a few high priority threads that execute nodes from the pool, in 
the correct order).

>
> - If there is one Dedicated worker per node, it's likely that 
> Dedicated workers will have to be created in great quantity since Web 
> Audio in many use cases is a very node-intensive system. The setup 
> overhead for dedicated workers may be too high.

That's my fear, too.

>
> - Dedicated workers imply postMessage-style communication via a 
> MessagePort, which suggests unnecessary communication overhead since 
> we don't need to allow passing of arbitrary data structures between 
> node workers and the rest of the environment. We can focus on audio 
> events and their results.

Well, I think that the message passing system could be a useful paradigm 
for contolling a script node from the main thread. For instance, it 
could work as a replacement for audio params until we have those figured 
out. It could also be useful for passing more complex data from the main 
thread to the script node (e.g. an impulse response).

I'm not saying that we must use Worker postMessage as-is - but I think 
that message passing in some form could be useful.

>
> - Going forward, Web Audio workers do not even need to communicate 
> with arbitrary other pieces of the browser environment. They only need 
> to communicate with the machinery running the audio graph.

I'm not convinced (see above), but I agree that it's a good thing if we 
can minimize the amount of non-audio-engine communication.

>
> So I suggest we at least consider a new flavor of Worker that is 
> tailored for WebAudio, or else move to a distinct object altogether.

+1

/Marcus

>
> …Joe
>
> On Oct 25, 2013, at 5:36 AM, Marcus Geelnard <mage@opera.com 
> <mailto:mage@opera.com>> wrote:
>> While the MSP solution is a possible path forward, I think that we 
>> should start with agreeing on a few requirements on the new script 
>> processor node before we can agree that it's the solution that we want.
>>
>> I think that there are a few differences in the basic ideas of how 
>> how MSP was envisioned to be used and how Web Audio is designed. 
>> Also, we have more experience now wrt actual use cases and feature 
>> requests.
>>
>> A few things that come to mind:
>>
>> * How well would the one-woker-per-node scale for many nodes (Nx 
>> script contexts)?
>>
>> * Is the one-worker-per-node design problematic for short-lived nodes 
>> (worker setup latency etc)?
>>
>> * From a client architecture point of view:
>>  - Can we support running these workers in high-priority threads to 
>> minimize latency?
>>  - Should the audio workers have special GC considerations in order 
>> to minimize worst-case latency?
>>
>> * As a consequence of the above points (and possibly others): Should 
>> the "workers" that we use be regular Web workers, dumbed down Web 
>> workers, or something else?
>>
>> * How about audio params, channel and input/output count control etc: 
>> Should they be in-scope or can we take that later?
>>
>>>
>>> I would like to see a group of people to come up with a detailed 
>>> proposal if there are remaining hurdles stopping us from including 
>>> this in the spec, and proposing a spec patch complete with 
>>> interfaces based on MSP.
>>>
>>> Also, may I suggest ROC should lead this? What say you, ROC?
>>
>> I'd like to see ROC head this group too.
>>
>> /Marcus
>>
>>>
>>> Thanks,
>>> --
>>> Olivier
>>>
>>>
>>>
>>>
>>>
>>> -----------------------------
>>> http://www.bbc.co.uk
>>> This e-mail (and any attachments) is confidential and
>>> may contain personal views which are not the views of the BBC unless 
>>> specifically stated.
>>> If you have received it in
>>> error, please delete it from your system.
>>> Do not use, copy or disclose the
>>> information in any way nor act in reliance on it and notify the sender
>>> immediately.
>>> Please note that the BBC monitors e-mails
>>> sent or received.
>>> Further communication will signify your consent to
>>> this.
>>> -----------------------------
>>>
>>
>>
>> -- 
>> Marcus Geelnard
>> Technical Lead, Mobile Infrastructure
>> Opera Software
>>
>>
>
> .          .   .    .  . ...Joe
>
> *Joe Berkovitz*
> President
>
> *Noteflight LLC*
> Boston, Mass.
> phone: +1 978 314 6271
> www.noteflight.com <http://www.noteflight.com>
> "Your music, everywhere"
>


-- 
Marcus Geelnard
Technical Lead, Mobile Infrastructure
Opera Software