[web-audio-api] `ScriptProcessorNode` is Unfit For Purpose (#253)

The following issue was raised by the W3C TAG as part of their [review of the Web Audio API](https://github.com/w3ctag/spec-reviews/blob/master/2013/07/WebAudio.md)

### `ScriptProcessorNode` is Unfit For Purpose

We harbor deep concerns about `ScriptProcessorNode` as currently defined. Notably:

 * As currently defined, these nodes _must_ run on the UI thread -- giving rise, it seems, to many of the admonitions not to use them in Section 15.3.*. This is deeply at odds with the rest of the architecture which seeks to keep processing _off_ of the main thread to ensure low-latency.
 * Since these functions run on the main thread, it's impossible to set execution constraints that might help reduce jitter; e.g., a different GC strategy or tight constraints on execution time slicing.

This can be repaired. Here's a stab at it:

```
[Constructor(DOMString scriptURL, optional unsigned long bufferSize)]
interface AudioWorker : Worker {

};

interface AudioProcessingEvent : Event {

    readonly attribute double playbackTime;

    transferrable attribute AudioBuffer buffer;

};

interface AudioWorkerGlobalScope : DedicatedWorkerGlobalScope {

  attribute EventHandler onaudioprocess;

};

interface ScriptProcessorNode : AudioNode {

  attribute EventHandler onaudioprocess;

  readonly attribute long bufferSize;

};

partial interface AudioContext {

  ScriptProcessorNode createScriptProcessor(
   DOMString scriptURL,
    optional unsigned long bufferSize = 0,
    optional unsigned long numberOfInputChannels = 2,
    optional unsigned long numberOfOutputChannels = 2);

}
```

The idea here is that to ensure low-latency processing, no copying of resulting buffers is done (using the Worker's [Transferrable](https://plus.sandbox.google.com/114636678211810483111/posts/isYunS2B6os) mechanism).

Scripts are loaded from external URLs and can control their inbound/outbound buffering with a constructor arguments.

Under this arrangement it's possible for the system to start to change the constraints that these scripts run under. GC can be turned off, runtime can be tightly controlled, and these scripts can even be run on the (higher priority) audio-processing thread.

All of this is necessary to ensure that scripts are not second-class citizens in the architecture; attractive nuisances which can't actually be used in the real world due to their predictable down-sides.

### ISSUE: Explaining ("De-sugaring") the `*Node` Types

On the back of a repaired `ScriptProcessorNode` definition, the spec should contain tight descriptions of the built-in library of the `AudioNode` subclasses; preferably in the form of script which could be executed in a `ScriptProcessorNode`.

*Obviously* we do not recommend that implementations lean on this sort of self-hosting for production systems. It is, however, clear that without such a detailed description of the expected algorithms, compatibility between implementations cannot be guaranteed, nor can conformance with the spec be meaningfully measured. This de-sugaring-as-spec-exercise would be helpful for the testing of the system and for users who will at a later time want to know _exactly_ what the system is expected to be doing for them.

We imagine an appendix of the current spec that includes such de-sugarings and test suites built on them.


---
Reply to this email directly or view it on GitHub:
https://github.com/WebAudio/web-audio-api/issues/253

Received on Thursday, 17 October 2013 12:25:54 UTC