W3C home > Mailing lists > Public > public-xg-audio@w3.org > December 2010

Re: Making Web Audio APIs More Competitive with native

From: Chris Rogers <crogers@google.com>
Date: Thu, 9 Dec 2010 11:12:03 -0800
Message-ID: <AANLkTinP0p__ee6wTRH1Z_1FBC-SeOpS3mZoG2z_uuoe@mail.gmail.com>
To: Noah Mendelsohn <nrm@arcanedomain.com>
Cc: public-xg-audio@w3.org, Doug Schepers <schepers@w3.org>
Hi Noah, I really appreciate your interest.  I'm doing my best to ensure
that the API is able to handle multi-channel, and other advanced use cases.
 Clearly, API-wise we don't want to lock ourselves into a corner and should
plan for the future.  That said, working with audio in a browser environment
presents a unique set of challenges and I don't expect that a JavaScript
developer is going to play with my API for a couple of weeks and come back
with Digital Audio Workstation software to rival something like Apple's
Logic Audio :)  These types of desktop applications have had years of
development by teams of engineers.  I try to present a balanced picture of
what I think is possible in my "Introduction" sections of my specification:

That said, I'm very excited about the possibilites of building this platform
and am looking forward to seeing some great applications built on it.


On Thu, Dec 9, 2010 at 5:58 AM, Noah Mendelsohn <nrm@arcanedomain.com>wrote:

> There is one more thing that occurs to me: it would be really great if some
> of the early implementations supported this sort of high quality
> multi-channel audio, e.g. supporting ASIO on Windows or whatever is the
> appropriate equivalent on Mac.  That would not only help validate that the
> API meets its goals, it would of course encourage early experimentation and
> innovation.
> Of course, I understand that doing such things is real work, and that the
> resources to do it may or may not actually be available.  I do think it
> would be very valuable, and as a user, I'd be really anxious to play with it
> if it were available.
> Thanks again.
> Noah
> On 12/8/2010 9:52 PM, Noah Mendelsohn wrote:
>> Chris,
>> Thank you for the quick and thoughtful response. At a lunchtime discussion
>> at the recent W3C plenary, I was led to believe that on current course and
>> speed the Web APIs would likely not have the low-latency >2 channel
>> capabilities I suggested. I'm delighted to hear from your note that they
>> probably will.
>> As I tried to say in my first note, I claim no expertise in the low-level
>> details of any of these APIs. So, I'll respectfully decline your request
>> that:
>>  > it would be helpful to have some concrete suggestions for improving the
>>  > API.
>> on the grounds that I'm not specifically aware of any shortcomings, and
>> don't necessarily have the expertise to suggest improvements even if there
>> were problems. I just want to know that you all are aiming for function
>> competitive with native, and it seems you are.
>> I don't know whether any future standardization activities relating to
>> these APIs will go through a formal requirements setting process, but if
>> so, I do recommend that the multi-channel DAW use case be considered.
>> Thank you again.
>> Noah
>> On 12/8/2010 1:37 PM, Chris Rogers wrote:
>>> Hi Noah, thanks for your input.
>>> On Wed, Dec 8, 2010 at 8:13 AM, Noah Mendelsohn <nrm@arcanedomain.com
>>> <mailto:nrm@arcanedomain.com>> wrote:
>>> Doug Schepers noticed some comments I had made on another list, and
>>> encouraged me to post them here.
>>> Basically, I'd like to suggest that a goal be set of making Web Audio
>>> APIs comparable in capability with OS-native APIs such as Core Audio
>>> [1], ASIO [2], or maybe Direct Sound [3].
>>> I'm very familiar with these APIs. I worked at Apple for seven years and,
>>> with my team-mates, was very involved in the design and implementation of
>>> the Core Audio (and related) APIs. You may find a similarity between the
>>> web audio API I'm proposing and the Audio Units architecture which is
>>> used
>>> as the audio plugin model for Mac OS X. Audio Units are used in many
>>> third-party desktop audio applications.
>>> Stated differently, I think a goal should be to enable the creation of
>>> DAW software, if not quite at the Pro Tools [4] level, then at least as
>>> capable and robust as REAPER [5], using Web client technologies.
>>> Similarly, it should be possible to implement audio mixing
>>> applications to control live performance with professional quality.
>>> I am not deeply expert in the details of any of the native APIs used
>>> for such DAWs, but even as a novice, I can see certain technical
>>> characteristics that should be enabled:
>>> * In the case where a multichannel digital audio interface such as MOTU
>>> [6]. M-Audio [7], Presonus [8], etc. is available, then Javascript
>>> should be able to get at all the streams.
>>> There is nothing about the web audio API which would prohibit this.
>>> * It should be possible to get at the full resolution that the device
>>> supports (e.g. 24 bit 192Khz 16 track sampling)
>>> The web audio engine uses 32bit floating-point data for internal
>>> processing
>>> just as many pro desktop audio applications do. These streams are
>>> available for processing in JavaScript, again at 32bit float resolution.
>>> * Simultaneous recording from some channels while driving playback on
>>> others.
>>> I haven't added audio input (for recording) into the specification yet,
>>> but
>>> as I've already responded in a previous email, the API is designed to be
>>> extendible to support this.
>>> * Low latency is essential: it should be possible to sync all record and
>>> playback streams with very little time skew.
>>> Achieving the lowest possible latency has been a top priority in the
>>> system
>>> I'm proposing and I have a section in my specification where I discuss
>>> it:
>>> http://chromium.googlecode.com/svn/trunk/samples/audio/specification/specification.html#Performance-section
>>> There may be other features needed. My point is that the capabilities
>>> described above are widely available with commodity hardware and audio
>>> software on laptop-grade personal computers. If the Web doesn't have the
>>> same capabilities, then it will be a 2nd class platform for audio
>>> applications. The demos I've seen of early implementations of Web audio
>>> APIS are very cool, but they seem to target a much simpler use case, I.e.
>>> of filtering or imaging simple stereo streams as they are played by the
>>> browser. Those don't seem to me to be sufficiently demanding if the Web
>>> is
>>> to be appropriately competitive with OS-native capabilities.
>>> The demos I've created were intended to be simple to illustrate the
>>> basic concepts of the API. Keep in mind that most of my time has
>>> been spent implementing the engine itself, so I haven't had time to
>>> create more elaborate demos.
>>> I do understand that the use of a language like Javascript may, at a
>>> given
>>> point in time and on a particular platform, limit the performance that
>>> can
>>> be achieved. Still, my intuition is that modern systems are fast enough
>>> to
>>> build a respectable DAW using Web technologies, and in any case the APIs
>>> should be designed from the start to scale to such applications.
>>> Noah, I would be interested in specific aspects of the API which you feel
>>> limit the possibility of creating such applications.
>>> I've created a complete specification:
>>> http://chromium.googlecode.com/svn/trunk/samples/audio/specification/specification.html
>>> And it would be helpful to have some concrete suggestions for improving
>>> the
>>> API.
>>> Best Regards,
>>> Chris
Received on Thursday, 9 December 2010 19:12:35 UTC

This archive was generated by hypermail 2.4.0 : Friday, 17 January 2020 19:37:59 UTC