Re: Aiding early implementations of the web audio API

Den 2012-05-21 19:40:10 skrev Chris Wilson <cwilso@google.com>:

> On Mon, May 21, 2012 at 4:44 AM, Marcus Geelnard <mage@opera.com> wrote:
>
>> However, a large part of the API seems to come into good use mostly for
>> "Musical Applications".
>
> I'm not sure I agree with this.  I think you'll find that a lot of
> environmental effects are easily replicated ("voice on a telephone", etc)
> are easily done by using filters and convolution.  It's certainly true  
> that
> there are a few areas that this is the case - e.g. Oscillator and
> DynamicsCompressor - but a lot of the other effects can make immersive
> sound environments in games a lot easier to create.

Yes, that's true. The filter and convolver nodes are certainly useful for  
game sound effects (underwater, cave, etc). Other nodes can also be used  
for creating interesting effects, but that's not the point.

My view on the issue is that the more complex the spec is, the more it  
will cost (spec work, implementation time, test suits, bug fixing,  
source/binary sizes, etc), and the more likely it is that we will have  
different behavior in different implementations, ranging from noticeable  
performance differences to noticeable differences in sound and behavior  
and possibly implementation-dependent corner case bugs etc.

Since all nodes can be implemented in JavaScript (most of them are even  
trivial), the only reason for using native nodes instead of JavaScript  
nodes is to improve performance.

Have any performance comparisons been made between the native nodes and  
their corresponding JavaScript implementations? I'm quite sure that native  
implementations will be faster (perhaps significantly in several cases),  
and I can also make some guesses as to which nodes would be actual  
performance bottle necks, but to what extent?

I guess what I'm getting at is: What is the minimal subset of the API that  
we *need* to be able to support the majority of use cases (in terms of the  
actual number of end users)? And on top of this, what nodes do we *have*  
to implement natively in order to get acceptable performance in 3D games,  
for example.

I also think it's worth considering exposing the most critical processing  
primitives (e.g. FFT) as JavaScript functions rather than AudioNodes. That  
would bring the performance gap down even more, and open up for even more  
interesting possibilities.

/Marcus


>> This leads me to believe that the JavaScript processing node will be  
>> very
>> important (for implementing custom effects and instruments, and possibly
>> even for creating effect libraries), while some native nodes (such as  
>> the Oscillator, BiquadFilterNode, DynamicsCompressorNode and DelayNode  
>> nodes) will not be used as much.
>>
>
> Actually, I have to disagree there - I think you'll find a TREMENDOUS
> amount of the processing to create custom processing and effects can be
> easily built on top of the primitives in the Web Audio proposal today; I
> think filters, delays and convolution will remove a lot of the needs for
> JavaScript nodes.
>
> As an example - for Google IO, I'm building a demo that does vocoding  
> using the Web Audio API.  In order to implement this, I've had to use  
> bandpass
> filters, lowpass filters, waveshapers, gain nodes in interesting ways,
> Oscillators (including a WaveTable oscillator, since those work  
> incredibly well as carrier sources for vocoders).  I went into the  
> project fairly
> naively (I own a vocoder rack unit, I understand the basic concept, but I
> had a lot of learning to do). This has prompted me - and I am NOT an  
> audio engineer, FWIW - to consider a bunch of the cases I find  
> interesting to
> think through, like AutoTune-like effects and harmonizers, and distortion
> effects to build guitar amp simulators, etc. - even aside from my  
> original passion, which is software synthesizers.  I'm fairly convinced  
> that most of the building blocks are there, and JavaScript nodes will be  
> needed only in a few cases.
>
> That doesn't mean that JavaScriptNodes are not definitely a critical  
> case, and should work well - and I think the first time I met Chris in  
> person (we work in different offices) I asked if he would add  
> Worker-based
> JavaScriptNode support.  I really would not expect developers to use it
> first, though - it's a great tool, but it's not the best tool for the  
> job a lot of the time.
>
> -Chris


-- 
Använder Operas banbrytande e-postklient: http://www.opera.com/mail/

Received on Tuesday, 22 May 2012 09:14:31 UTC