Re: Aiding early implementations of the web audio API

On Mon, May 21, 2012 at 4:35 PM, Alessandro Saccoia <
alessandro.saccoia@gmail.com> wrote:

> I would like to lump in some thoughts I had while using the API:
> - the specification of the Biquad node leaves a complete freedom to the
> implementation about the way the filter coefficients are computed. For
> Butterworth filters, the RBJ cookbook formulas would work,


Yes, we will spec this down to the exact filter coefficients (Butterworth,
etc.) for the filter types defined.  I think we already have with the RBJ
formulas, but it has been requested to bring the formulas directly into the
spec...


> but in other cases one could need a different type of filter.  It would
> then be nice to expose the coefficients, so that the developers can twiddle
> with them without the need of recurring to custom filters implemented in
> Javascript Nodes.
>

Yes, I agree!  I just haven't yet put something like that in the spec
because people are still trying to digest what's in there today.


> - for now, the only way to apply envelopes is to use the family method
> SetValuAtTime and its relatives… and I guess that there's a bit of overhead
> to handle all that scheduling when the envelopes are needed to be sample
> accurate.


In WebKit, we've gone to great lengths to optimize these envelopes.


> This makes also the programming interface a bit awkward. An example of
> this is in the Granular Effects example.
>

I'm not sure which part you don't like.  In that example we create a custom
curve for the grain window and then simply call setValueCurveAtTime().  I
think it's pretty concise (few lines of JS) and is flexible, allowing
arbitrary curves.


> - With the current specifications I don't think it could be possible to
> build a FM synthesizer, because there is no way to know if when setting the
> Oscillator frequency the phase will be reset or not.
>

It's definitely possible.  In fact, I've been playing around with some FM
demos and getting good results.  The demos are not quite ready to show, but
hopefully I can show them quite soon.


> - To do complex routings a number of Gain nodes have to be used just to do
> interconnections.

A pure node that doesn't do any memory copying or processing could be quite
> useful.
>

I'm not sure what you mean by this.  Gain nodes are used if you need to
change the gain of something, otherwise fan-in (connections from multiple
inputs) works automatically with implicit summing.


> - native support of the FFT with an interface like the one in pure data or
> max msp would take minimal effort but give great joy to some people, and in
> this case a native implementation would have a big performance gain
>

I'm open to suggestions if you can give some more detail.


> - oversampling and decimation would be nice to have too, not just to avoid
> aliasing in some audio effects, but also to let parts of the processing
> chain to work at a slower rate
>

We've already discussed having different parts of the graph run at
different rates.  I think it could be added in the future, but for now it's
probably better to keep things simple.  Otherwise, we have to worry about
connections between nodes running at different rates.  It's possible to
define, but this is quite a specialized use case.

By the way, I'm very supportive of having the WaveShaperNode optionally
*internally* implement the non-linear distortion at higher-sample rates to
avoid aliasing.  I think this will be essential for high-quality distortion
effects.

I could be wrong on multiple things here as I just started using this api,
> so please correct me if I am wrong!
>

Thanks for checking things out!

Cheers,
Chris


> alessandro
>
> On May 21, 2012, at 7:40 PM, Chris Wilson wrote:
>
> > On Mon, May 21, 2012 at 4:44 AM, Marcus Geelnard <mage@opera.com> wrote:
> > However, a large part of the API seems to come into good use mostly for
> "Musical Applications".
> >
> > I'm not sure I agree with this.  I think you'll find that a lot of
> environmental effects are easily replicated ("voice on a telephone", etc)
> are easily done by using filters and convolution.  It's certainly true that
> there are a few areas that this is the case - e.g. Oscillator and
> DynamicsCompressor - but a lot of the other effects can make immersive
> sound environments in games a lot easier to create.
> >
> > This leads me to believe that the JavaScript processing node will be
> very important (for implementing custom effects and instruments, and
> possibly even for creating effect libraries), while some native nodes (such
> as the Oscillator, BiquadFilterNode, DynamicsCompressorNode and DelayNode
> nodes) will not be used as much.
> >
> > Actually, I have to disagree there - I think you'll find a TREMENDOUS
> amount of the processing to create custom processing and effects can be
> easily built on top of the primitives in the Web Audio proposal today; I
> think filters, delays and convolution will remove a lot of the needs for
> JavaScript nodes.
> >
> > As an example - for Google IO, I'm building a demo that does vocoding
> using the Web Audio API.  In order to implement this, I've had to use
> bandpass filters, lowpass filters, waveshapers, gain nodes in interesting
> ways, Oscillators (including a WaveTable oscillator, since those work
> incredibly well as carrier sources for vocoders).  I went into the project
> fairly naively (I own a vocoder rack unit, I understand the basic concept,
> but I had a lot of learning to do). This has prompted me - and I am NOT an
> audio engineer, FWIW - to consider a bunch of the cases I find interesting
> to think through, like AutoTune-like effects and harmonizers, and
> distortion effects to build guitar amp simulators, etc. - even aside from
> my original passion, which is software synthesizers.  I'm fairly convinced
> that most of the building blocks are there, and JavaScript nodes will be
> needed only in a few cases.
> >
> > That doesn't mean that JavaScriptNodes are not definitely a critical
> case, and should work well - and I think the first time I met Chris in
> person (we work in different offices) I asked if he would add Worker-based
> JavaScriptNode support.  I really would not expect developers to use it
> first, though - it's a great tool, but it's not the best tool for the job a
> lot of the time.
> >
> > -Chris
>
>
>