Web Audio API questions and comments

Hi all,
I've had a quick run through the specification again and had a couple
of questions and observations which I thought might be helpful.  I'm
afraid it might be covering quite a few areas in one email, so feel
free to split this out into multiple topics if it's easier to follow.
Also apologies if I'm going over things which have already been
discussed; I've tried my best to keep up with the email traffic, but I
think I've probably missed or forgotten a few things.

Firstly, though a quick thanks to all involved - the spec is looking a
lot more awesome than when I last had a good look through.

- Block size

I know this is a fairly broad architectural issue, but how strong is
the case for processing blocks of samples rather than single samples?
The spec states that it's for performance reasons - is the difference
in performance quantified?  I know that the majority of current
systems use blocks, but there are notable exceptions (ChucK [1] is the
most prominent one that I know of) which work with one sample at a
time.  The practical advantage of single samples is that it allows for
very short feedback loops.  For an example of this in practice,
Datorro's figure-of-8 reverb algorithm [2] uses a feedback loop of 107
samples, which would be impossible without resorting to the
reimplementation of a number of existing nodes in JavaScript.

- AudioNode cycles with JavaScriptAudioNodes

Given that JavaScriptAudioNodes have an inherent delay built in they
should be fine to use in feedback loops as well as DelayNodes I think.
 Is this correct?

- Control rate AudioParams and interpolation

Will there ever be a use for interpolation between values with a
control rate AudioParam rather than skipping to the next value at the
start of each block?  The Supercollider documentation [3] mentions
that this technique is used in some UGens which seems plausible, but
I'm not clear on when or why this is appropriate.  Does something like
this need specifying?

- AudioParam setValueCurveAtTime with an Array parameter

Should this be able to take a normal JavaScript array as a parameter?
For simple fixed envelopes this method seems simpler than having a
number of linearRampToValueAtTime calls, but the array will only
contain a few values so creating a Float32Array seems a bit like
overkill.

- AudioGainNode dezippering

Can this at least be optional?  If I'm using an AudioGainNode to scale
an audio node so it can control an AudioParam (for example to act as
an lfo), then I don't want the output to be filtered in any way.
Although a better solution may be:

- Operator nodes, AudioParam nodes

Can we have separate nodes for the basic mathematical operators (add,
subtract, divide, multiply and modulo), and a way of having the output
of an AudioParam as a signal?  This would allow all the flexibility
needed for scaling, offsetting and combining signals in order to
control parameters.  I know a bit of trickery can make stuff like this
possible at the moment, and it's trivial to implement in JavaScript,
but it seems like core functionality to me.

- Tapping DelayNodes

At the moment it's only possible to tap a delay line at the beginning,
and the read position cannot be modulated.  This makes it pretty much
impossible to implement effects such as pitch shifting, chorus,
phasing, flanging and modulated reverbs, all of which rely on either
multiple taps or modulated taps.  It would be nice to have something
similar to Supercollider's BufRd [4] and BufWr [5] with DelayNode
built as a layer on top of this.  Also AudioBufferSourceNode could be
a layer on top of a BufRd equivalent.

- AudioBufferSourceNode playbackState change event

Would it be useful to have an event fired when the
AudioBufferSourceNode's state changes?  I can't think of anything off
the top of my head, but it seems like it could be useful for some
applications maybe?

- JavaScriptAudioNode buffer size

Is there a technical limitation as to why the buffer size is limited
to these values, or would just any power of 2 suffice, maybe with a
note advising a minimum value?  Will using Workers make it possible to
use shorter buffers?  Also, I know this has been discussed a few times
- can I add a +1 on allowing JavaScriptAudioNodes access to
AudioParams.

- AudioPannerNode

Having never done any work in 3D sound I find this all a bit
intimidating.  Is there any chance of something simpler built on top
of this for those of us who want sound to come out of the left
speaker, the right speaker, or some combination of the two?

- RealTimeAnalyserNode

This seems strange to me - the functionality could be really useful,
but it seems focused very narrowly on creating visualisations.  I
think a nicer solution would be to have separate FFT and IFFT nodes so
frequency domain effects could be integrated into the processing
chain, and then a separate node which allows access to the FFT or
waveform data depending on where in the graph it is inserted.  So for
visualisations you would have an AudioNode connected to an FFTNode,
connected to a BufferYoinkerNode.

A couple more small points on this.  Firstly the name is very
non-specific - I think it should probably at least describe what
analysis is being done.  It would also be good to allow windowing
functions to be supplied, or at least specify which windowing function
should be used.

- DynamicsCompressorNode sidechaining and lookahead

I'm not sure if these are a bit specialised, a bit of a dark art, or
both, but they are both common and fairly well defined features of
compressors which may be useful.  I could see sidechaining being
especially useful for ducking in broadcast applications.

- First order filters

It seems a little bit strange that second order (Biquad) filters are
provided as standard, whereas first order filters aren't.  Again, this
is pretty trivial in JavaScript, I was just wondering what the reason
for their omission is?  First order filters are fairly widely used in
a number of common audio algorithms, from a basic envelope follower
through to providing damping in reverbs.

Hope some of this is helpful,
Thanks again,
Joe



[1] http://chuck.cs.princeton.edu/
[2] https://ccrma.stanford.edu/~dattorro/EffectDesignPart1.pdf, p. 662
[3] http://doc.sccode.org/Tutorials/Getting-Started/11-Busses.html
[4] http://doc.sccode.org/Classes/BufRd.html
[5] http://doc.sccode.org/Classes/BufWr.html

Received on Tuesday, 19 June 2012 14:40:33 UTC