Re: ScriptProcessorNode off the main thread (Was: Questioning the current direction of the Web Audio API)

2013-10-25 11:09, Olivier Thereaux skrev:
> On 25 Oct 2013, at 08:04, Jussi Kalliokoski <jussi.kalliokoski@gmail.com> wrote:
>
>>> So where can we go from here? I think we pretty clearly need a version of ScriptProcessorNode that does JS audio processing off the main thread. The obvious and expedient approach would be to have it dispatch messages to a Worker just like MSP did. It should work with Float32 typed arrays rather than AudioBuffers and be designed to avoid requiring buffer copies. I think that we can arrange for very-short-running JS audio processing to run synchronously during Web Audio processing (by having that realtime thread wait for the JS processing to complete, with a short timeout in case it's slow). We need to define what happens if the processing is too slow; ideally that should just cause a latency increase, if we're not in a true overload situation.
>>>
>>> That should help a lot for a lot of use-cases. If it turns out to be an incomplete solution, we'll have a lot more data to inform our next move.
>>>
>> I highly agree with this. In my opinion, this is the sane approach of solving problems little by little, gathering data of the real pain points as we go, and pivoting as needed.
> +1
>
> We've spent a lot of time in the past discussing worker-only versus main-thread-only and not as much trying to figure out how to make it work. We did, however, get to a resolution to enable ScriptProcessorNode with workers:
>
> https://github.com/WebAudio/web-audio-api/issues/113?source=cc#issuecomment-24244840
>
> Anyone has a clear picture of whatever is blocking the inclusion of this resolution into the spec? Or is it just a byproduct of our editorial transition over spring/summer?

While the MSP solution is a possible path forward, I think that we 
should start with agreeing on a few requirements on the new script 
processor node before we can agree that it's the solution that we want.

I think that there are a few differences in the basic ideas of how how 
MSP was envisioned to be used and how Web Audio is designed. Also, we 
have more experience now wrt actual use cases and feature requests.

A few things that come to mind:

* How well would the one-woker-per-node scale for many nodes (Nx script 
contexts)?

* Is the one-worker-per-node design problematic for short-lived nodes 
(worker setup latency etc)?

* From a client architecture point of view:
   - Can we support running these workers in high-priority threads to 
minimize latency?
   - Should the audio workers have special GC considerations in order to 
minimize worst-case latency?

* As a consequence of the above points (and possibly others): Should the 
"workers" that we use be regular Web workers, dumbed down Web workers, 
or something else?

* How about audio params, channel and input/output count control etc: 
Should they be in-scope or can we take that later?

>
> I would like to see a group of people to come up with a detailed proposal if there are remaining hurdles stopping us from including this in the spec, and proposing a spec patch complete with interfaces based on MSP.
>
> Also, may I suggest ROC should lead this? What say you, ROC?

I'd like to see ROC head this group too.

/Marcus

>
> Thanks,
> --
> Olivier
>
>
>
>
>
> -----------------------------
> http://www.bbc.co.uk
> This e-mail (and any attachments) is confidential and
> may contain personal views which are not the views of the BBC unless specifically stated.
> If you have received it in
> error, please delete it from your system.
> Do not use, copy or disclose the
> information in any way nor act in reliance on it and notify the sender
> immediately.
> Please note that the BBC monitors e-mails
> sent or received.
> Further communication will signify your consent to
> this.
> -----------------------------
>


-- 
Marcus Geelnard
Technical Lead, Mobile Infrastructure
Opera Software

Received on Friday, 25 October 2013 09:37:24 UTC