Re: Web Audio API Proposal

On Tue, Jun 15, 2010 at 3:34 PM, Robert O'Callahan <robert@ocallahan.org>wrote:

> On Wed, Jun 16, 2010 at 7:49 AM, Chris Rogers <crogers@google.com> wrote:
>
>>  Including a richer set of higher-level graphics APIs in canvas and WebGL
>> is important not only for the obvious performance issues, but also because
>> the APIs are higher-level and more directly useful to the javascript
>> developer.  Similarly, for audio we face the same types of performance
>> issues, but equally important is the need for a higher-level API.
>
>
> In WebGL we actually provide a relatively low-level API and JS developers
> pretty much have to use libraries to build real applications.
>
In general I think we should expect developers to have to use libraries to
> build complex applications. Hard-coding every desirable effect or feature
> directly into every browser is not sustainable. The Web has already gone far
> too far down that road.
>

In some respects WebGL is a low-level API, but consider if it didn't exist
and you had to write all the javascript code to implement an OpenGL software
renderer.  Here, you would have to blit every single pixel one at a time
with javascript.  It would be immensely complicated.  Looked at from this
perspective, WebGL is a high-level API.  I think the audio API I'm proposing
is roughly at this level.



>  I don't believe that simply moving the JS processing to a web worker is
>> going to solve all of the glitching and latency issues.
>
>
> What fundamental limitations to do you see?
>

I think we should explore ways to try to tackle these issues with JS
processing.  Maybe web workers is a good start, but I think there will be
issues with thread priorities, locks being taken, blocking calls in the web
worker thread, garbage collection...  Also I could be wrong about this, but
having javascript developers put all their audio code in a web worker may be
a bit cumbersome for them.


>
> Some effects *can* be made fast enough.  But some very common cases can't
>> be made fast enough
>>
>
> Which cases?
>

I'm thinking of games where multiple sources are being spatialized.
 Convolution effects are another case.  Also I think there would be cases in
musical applications with many notes playing, or multiple sources with
multiple effects per source.

Chris