Re: Aiding early implementations of the web audio API

I would like to lump in some thoughts I had while using the API:
- the specification of the Biquad node leaves a complete freedom to the implementation about the way the filter coefficients are computed. For Butterworth filters, the RBJ cookbook formulas would work, but in other cases one could need a different type of filter.  It would then be nice to expose the coefficients, so that the developers can twiddle with them without the need of recurring to custom filters implemented in Javascript Nodes.
- for now, the only way to apply envelopes is to use the family method SetValuAtTime and its relatives… and I guess that there's a bit of overhead to handle all that scheduling when the envelopes are needed to be sample accurate. This makes also the programming interface a bit awkward. An example of this is in the Granular Effects example.
- With the current specifications I don't think it could be possible to build a FM synthesizer, because there is no way to know if when setting the Oscillator frequency the phase will be reset or not. 
- To do complex routings a number of Gain nodes have to be used just to do interconnections. A pure node that doesn't do any memory copying or processing could be quite useful.
- native support of the FFT with an interface like the one in pure data or max msp would take minimal effort but give great joy to some people, and in this case a native implementation would have a big performance gain
- oversampling and decimation would be nice to have too, not just to avoid aliasing in some audio effects, but also to let parts of the processing chain to work at a slower rate
I could be wrong on multiple things here as I just started using this api, so please correct me if I am wrong!
alessandro

On May 21, 2012, at 7:40 PM, Chris Wilson wrote:

> On Mon, May 21, 2012 at 4:44 AM, Marcus Geelnard <mage@opera.com> wrote:
> However, a large part of the API seems to come into good use mostly for "Musical Applications".
> 
> I'm not sure I agree with this.  I think you'll find that a lot of environmental effects are easily replicated ("voice on a telephone", etc) are easily done by using filters and convolution.  It's certainly true that there are a few areas that this is the case - e.g. Oscillator and DynamicsCompressor - but a lot of the other effects can make immersive sound environments in games a lot easier to create.
> 
> This leads me to believe that the JavaScript processing node will be very important (for implementing custom effects and instruments, and possibly even for creating effect libraries), while some native nodes (such as the Oscillator, BiquadFilterNode, DynamicsCompressorNode and DelayNode nodes) will not be used as much.
> 
> Actually, I have to disagree there - I think you'll find a TREMENDOUS amount of the processing to create custom processing and effects can be easily built on top of the primitives in the Web Audio proposal today; I think filters, delays and convolution will remove a lot of the needs for JavaScript nodes.
> 
> As an example - for Google IO, I'm building a demo that does vocoding using the Web Audio API.  In order to implement this, I've had to use bandpass filters, lowpass filters, waveshapers, gain nodes in interesting ways, Oscillators (including a WaveTable oscillator, since those work incredibly well as carrier sources for vocoders).  I went into the project fairly naively (I own a vocoder rack unit, I understand the basic concept, but I had a lot of learning to do). This has prompted me - and I am NOT an audio engineer, FWIW - to consider a bunch of the cases I find interesting to think through, like AutoTune-like effects and harmonizers, and distortion effects to build guitar amp simulators, etc. - even aside from my original passion, which is software synthesizers.  I'm fairly convinced that most of the building blocks are there, and JavaScript nodes will be needed only in a few cases.
> 
> That doesn't mean that JavaScriptNodes are not definitely a critical case, and should work well - and I think the first time I met Chris in person (we work in different offices) I asked if he would add Worker-based JavaScriptNode support.  I really would not expect developers to use it first, though - it's a great tool, but it's not the best tool for the job a lot of the time.
> 
> -Chris

Received on Monday, 21 May 2012 23:35:56 UTC