Re: Making Web Audio APIs More Competitive with native

On Fri, Dec 10, 2010 at 8:39 PM, Kumar <srikumarks@gmail.com> wrote:

> Hi Chris,
>
> Yes, you're right about the immaturity of OpenSL ES. It is a
> relatively recent spec indeed. OpenAL (together with EFX) is
> a more mature candidate.
>

I implemented the underlying engine for OpenAL on Mac OS X.  Beyond the very
base-level API, there are significant differences in the features and even
the API between platforms.  With help from WebKit engineers at Apple and
others on this list, I've tried to make my proposed API a somewhat more
compelling alternative which supports a superset of the features of OpenAL,
while addressing the current serious compatibility problems it has.  One
problem it has is the way that environmental spaces (room effects) are
handled.  This same issue also exists with OpenSL.  I wrote about this in
section 12 of my specification (Linear Effects using Convolution -
subsection "Motivation for use as a Standard"):
http://chromium.googlecode.com/svn/trunk/samples/audio/specification/specification.html#Spatialization-section

The symmetry I was referring to is more about the broad
> principles behind considering existing APIs ... three of them -
>
> 1) Reducing developer cognitive load in the long run,
>

I'm not really sure what this means.



> 2) Maximizing audio hardware access in the long run and
>

Browsers already have access to the basic audio hardware to support <audio>
and <video> elements.


3) Reducing browser implementation burden in the long run.
>
> On various platforms, we have OpenAL, Core Audio,
> DirectSound, OSS, ALSA, ESD, JACK, LADSPA. Of these,
> possibly OpenAL has the widest base. If we're to look a couple
> of years into the future, I can't help wonder if OpenSL ES
> will be the more established native API. It is certainly getting
> on to Android [http://developer.android.com/sdk/ndk/overview.html].
>
> If OpenSL ES is prevalent on mobile platforms and it can
> wrap around existing platform-specific audio APIs and codecs,
> then having its object model directly available via Javascript
> seems likely to maximize the audio hardware access across the
> board. Given that platform vendors will provide the implementation,
> adopting it will reduce the implementation burden on WebKit/Mozilla,
> and reduce the need for developers to work with one more audio API
> in the mix.


Kumar, I think I simply disagree that OpenSL would be the path of least
resistance to implement an audio API in the browser.  As far as WebKit is
concerned, I've just landed a complete implementation of my API which is set
to work with Apple's Safari and Google's Chrome.  Something you may not
realize is that Android, iOS, and other major mobile platforms use WebKit
for their web implementation.  Thus there is substantial finished work that
potentially could be leveraged there.  Having looked over the complexity of
the OpenSL specification and having experience in the WebKit codebase
myself, I believe it would be easier to adopt my implementation.  As far as
other browser codebases go, I don't think there's any compelling evidence
that an OpenSL implementation would be easier.

One point which is important to realize is that OpenSL is a low-level C API
which is difficult and complex to work with, even for the simplest audio
tasks.  For examples of this complexity, please see the numerous code
examples in the OpenSL specification.  The style and low-level nature of the
API also would, in my opinion, make it very difficult to adapt to a
JavaScript version of the API, with major changes being required.  In my
proposed API, great care has been taken to make it as powerful as possible,
while reducing the complexity of the code which JavaScript developers have
to write.

Lastly, despite the complexity of OpenSL (their specification is 555 pages
long!), it does not deliver all of the features of my API.  Their API does
not support fully modular routing, and instead has a rather limited fixed
single-send architecture.  It is not easily scalable due to its limited
modularity.  Also, it supports a rather crude model for spatialization and
environmental effects.  To be fair, it does support DLS sample-banks and a
built-in sequencer for MIDI files.  But this is a feature which can be
handled in other ways in my API.

Chris