Re: Questioning the current direction of the Web Audio API

[Let me note for historical purposes that there was a window of opportunity
to promote a more JS-aligned solution over Web Audio, but we missed that
window. That was partly my fault for not getting an implementation of
MediaStream Processing into Firefox early enough, but I also had
practically no support in this group helping me push back against the
Webkit "this is what we're shipping" juggernaut.]

It's important to keep in mind when comparing Web APIs to your favourite
native audio framework that on the Web, we have to handle untrusted
(possibly malicious, very likely poorly written) JS code. That's one reason
why running user JS audio processing code on a realtime audio processing
thread is a rather trickier proposition for us than for your native audio
framework. (And please don't play the "but MY code is perfect" card, it's
not helpful.)

So where can we go from here? I think we pretty clearly need a version of
ScriptProcessorNode that does JS audio processing off the main thread. The
obvious and expedient approach would be to have it dispatch messages to a
Worker just like MSP did. It should work with Float32 typed arrays rather
than AudioBuffers and be designed to avoid requiring buffer copies. I think
that we can arrange for very-short-running JS audio processing to run
synchronously during Web Audio processing (by having that realtime thread
wait for the JS processing to complete, with a short timeout in case it's
slow). We need to define what happens if the processing is too slow;
ideally that should just cause a latency increase, if we're not in a true
overload situation.

That should help a lot for a lot of use-cases. If it turns out to be an
incomplete solution, we'll have a lot more data to inform our next move.

Rob
-- 
Jtehsauts  tshaei dS,o n" Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.r"t sS?o  Whhei csha iids  teoa
stiheer :p atroa lsyazye,d  'mYaonu,r  "sGients  uapr,e  tfaokreg iyvoeunr,
'm aotr  atnod  sgaoy ,h o'mGee.t"  uTph eann dt hwea lmka'n?  gBoutt  uIp
waanndt  wyeonut  thoo mken.o w  *
*