Re: Making Web Audio APIs More Competitive with native

 Hi Noah, thanks for your input.

On Wed, Dec 8, 2010 at 8:13 AM, Noah Mendelsohn <nrm@arcanedomain.com>wrote:

> Doug Schepers noticed some comments I had made on another list, and
> encouraged me to post them here.
>
> Basically, I'd like to suggest that a goal be set of making Web Audio APIs
> comparable in capability with OS-native APIs such as Core Audio [1], ASIO
> [2], or maybe Direct Sound [3].
>

I'm very familiar with these APIs.  I worked at Apple for seven years and,
with my team-mates, was very involved in the design and implementation of
the Core Audio (and related) APIs.  You may find a similarity between the
web audio API I'm proposing and the Audio Units architecture which is used
as the audio plugin model for Mac OS X.  Audio Units are used in many
third-party desktop audio applications.


> Stated differently, I think a goal should be to enable the creation of DAW
> software, if not quite at the Pro Tools [4] level, then at least as capable
> and robust as REAPER [5], using Web client technologies.  Similarly, it
> should be possible to implement audio mixing applications to control live
> performance with professional quality.
>
> I am not deeply expert in the details of any of the native APIs used for
> such DAWs, but even as a novice, I can see certain technical characteristics
> that should be enabled:
>
> * In the case where a multichannel digital audio interface such as MOTU
> [6]. M-Audio [7], Presonus [8], etc. is available, then Javascript should be
> able to get at all the streams.
>

There is nothing about the web audio API which would prohibit this.


>
> * It should be possible to get at the full resolution that the device
> supports (e.g. 24 bit 192Khz 16 track sampling)
>

The web audio engine uses 32bit floating-point data for internal processing
just as many pro desktop audio applications do.  These streams are available
for processing in JavaScript, again at 32bit float resolution.


>
> * Simultaneous recording from some channels while driving playback on
> others.
>

I haven't added audio input (for recording) into the specification yet, but
as I've already responded in a previous email, the API is designed to be
extendible to support this.

* Low latency is essential: it should be possible to sync all record and
playback streams with very little time skew.

Achieving the lowest possible latency has been a top priority in the system
I'm proposing and I have a section in my specification where I discuss it:
http://chromium.googlecode.com/svn/trunk/samples/audio/specification/specification.html#Performance-section



There may be other features needed.  My point is that the capabilities
described above are widely available with commodity hardware and audio
software on laptop-grade personal computers.  If the Web doesn't have the
same capabilities, then it will be a 2nd class platform for audio
applications.  The demos I've seen of early implementations of Web audio
APIS are very cool, but they seem to target a much simpler use case, I.e. of
filtering or imaging simple stereo streams as they are played by the
browser.  Those don't seem to me to be sufficiently demanding if the Web is
to be appropriately competitive with OS-native capabilities.

   The demos I've created were intended to be simple to illustrate the basic
concepts of the API.  Keep in mind that most of my time has
   been spent implementing the engine itself, so I haven't had time to
create more elaborate demos.



I do understand that the use of a language like Javascript may, at a given
point in time and on a particular platform, limit the performance that can
be achieved.  Still, my intuition is that modern systems are fast enough to
build a respectable DAW using Web technologies, and in any case the APIs
should be designed from the start to scale to such applications.

Noah, I would be interested in specific aspects of the API which you feel
limit the possibility of creating such applications.
I've created a complete specification:
http://chromium.googlecode.com/svn/trunk/samples/audio/specification/specification.html

And it would be helpful to have some concrete suggestions for improving the
API.

Best Regards,
Chris