Re: Making Web Audio APIs More Competitive with native

Hi Noah, I really appreciate your interest.  I'm doing my best to ensure
that the API is able to handle multi-channel, and other advanced use cases.
 Clearly, API-wise we don't want to lock ourselves into a corner and should
plan for the future.  That said, working with audio in a browser environment
presents a unique set of challenges and I don't expect that a JavaScript
developer is going to play with my API for a couple of weeks and come back
with Digital Audio Workstation software to rival something like Apple's
Logic Audio :)  These types of desktop applications have had years of
development by teams of engineers.  I try to present a balanced picture of
what I think is possible in my "Introduction" sections of my specification:
http://chromium.googlecode.com/svn/trunk/samples/audio/specification/specification.html#introduction

That said, I'm very excited about the possibilites of building this platform
and am looking forward to seeing some great applications built on it.

Chris

On Thu, Dec 9, 2010 at 5:58 AM, Noah Mendelsohn <nrm@arcanedomain.com>wrote:

> There is one more thing that occurs to me: it would be really great if some
> of the early implementations supported this sort of high quality
> multi-channel audio, e.g. supporting ASIO on Windows or whatever is the
> appropriate equivalent on Mac.  That would not only help validate that the
> API meets its goals, it would of course encourage early experimentation and
> innovation.
>
> Of course, I understand that doing such things is real work, and that the
> resources to do it may or may not actually be available.  I do think it
> would be very valuable, and as a user, I'd be really anxious to play with it
> if it were available.
>
> Thanks again.
>
> Noah
>
>
> On 12/8/2010 9:52 PM, Noah Mendelsohn wrote:
>
>> Chris,
>>
>> Thank you for the quick and thoughtful response. At a lunchtime discussion
>> at the recent W3C plenary, I was led to believe that on current course and
>> speed the Web APIs would likely not have the low-latency >2 channel
>> capabilities I suggested. I'm delighted to hear from your note that they
>> probably will.
>>
>> As I tried to say in my first note, I claim no expertise in the low-level
>> details of any of these APIs. So, I'll respectfully decline your request
>> that:
>>
>>  > it would be helpful to have some concrete suggestions for improving the
>>  > API.
>>
>> on the grounds that I'm not specifically aware of any shortcomings, and
>> don't necessarily have the expertise to suggest improvements even if there
>> were problems. I just want to know that you all are aiming for function
>> competitive with native, and it seems you are.
>>
>> I don't know whether any future standardization activities relating to
>> these APIs will go through a formal requirements setting process, but if
>> so, I do recommend that the multi-channel DAW use case be considered.
>>
>> Thank you again.
>>
>> Noah
>>
>>
>> On 12/8/2010 1:37 PM, Chris Rogers wrote:
>>
>>> Hi Noah, thanks for your input.
>>>
>>> On Wed, Dec 8, 2010 at 8:13 AM, Noah Mendelsohn <nrm@arcanedomain.com
>>> <mailto:nrm@arcanedomain.com>> wrote:
>>>
>>> Doug Schepers noticed some comments I had made on another list, and
>>> encouraged me to post them here.
>>>
>>> Basically, I'd like to suggest that a goal be set of making Web Audio
>>> APIs comparable in capability with OS-native APIs such as Core Audio
>>> [1], ASIO [2], or maybe Direct Sound [3].
>>>
>>>
>>> I'm very familiar with these APIs. I worked at Apple for seven years and,
>>> with my team-mates, was very involved in the design and implementation of
>>> the Core Audio (and related) APIs. You may find a similarity between the
>>> web audio API I'm proposing and the Audio Units architecture which is
>>> used
>>> as the audio plugin model for Mac OS X. Audio Units are used in many
>>> third-party desktop audio applications.
>>>
>>>
>>> Stated differently, I think a goal should be to enable the creation of
>>> DAW software, if not quite at the Pro Tools [4] level, then at least as
>>> capable and robust as REAPER [5], using Web client technologies.
>>> Similarly, it should be possible to implement audio mixing
>>> applications to control live performance with professional quality.
>>>
>>> I am not deeply expert in the details of any of the native APIs used
>>> for such DAWs, but even as a novice, I can see certain technical
>>> characteristics that should be enabled:
>>>
>>> * In the case where a multichannel digital audio interface such as MOTU
>>> [6]. M-Audio [7], Presonus [8], etc. is available, then Javascript
>>> should be able to get at all the streams.
>>>
>>>
>>> There is nothing about the web audio API which would prohibit this.
>>>
>>>
>>> * It should be possible to get at the full resolution that the device
>>> supports (e.g. 24 bit 192Khz 16 track sampling)
>>>
>>>
>>> The web audio engine uses 32bit floating-point data for internal
>>> processing
>>> just as many pro desktop audio applications do. These streams are
>>> available for processing in JavaScript, again at 32bit float resolution.
>>>
>>>
>>> * Simultaneous recording from some channels while driving playback on
>>> others.
>>>
>>>
>>> I haven't added audio input (for recording) into the specification yet,
>>> but
>>> as I've already responded in a previous email, the API is designed to be
>>> extendible to support this.
>>>
>>> * Low latency is essential: it should be possible to sync all record and
>>> playback streams with very little time skew.
>>>
>>> Achieving the lowest possible latency has been a top priority in the
>>> system
>>> I'm proposing and I have a section in my specification where I discuss
>>> it:
>>>
>>> http://chromium.googlecode.com/svn/trunk/samples/audio/specification/specification.html#Performance-section
>>>
>>>
>>>
>>>
>>> There may be other features needed. My point is that the capabilities
>>> described above are widely available with commodity hardware and audio
>>> software on laptop-grade personal computers. If the Web doesn't have the
>>> same capabilities, then it will be a 2nd class platform for audio
>>> applications. The demos I've seen of early implementations of Web audio
>>> APIS are very cool, but they seem to target a much simpler use case, I.e.
>>> of filtering or imaging simple stereo streams as they are played by the
>>> browser. Those don't seem to me to be sufficiently demanding if the Web
>>> is
>>> to be appropriately competitive with OS-native capabilities.
>>>
>>> The demos I've created were intended to be simple to illustrate the
>>> basic concepts of the API. Keep in mind that most of my time has
>>> been spent implementing the engine itself, so I haven't had time to
>>> create more elaborate demos.
>>>
>>>
>>>
>>> I do understand that the use of a language like Javascript may, at a
>>> given
>>> point in time and on a particular platform, limit the performance that
>>> can
>>> be achieved. Still, my intuition is that modern systems are fast enough
>>> to
>>> build a respectable DAW using Web technologies, and in any case the APIs
>>> should be designed from the start to scale to such applications.
>>>
>>> Noah, I would be interested in specific aspects of the API which you feel
>>> limit the possibility of creating such applications.
>>> I've created a complete specification:
>>>
>>> http://chromium.googlecode.com/svn/trunk/samples/audio/specification/specification.html
>>>
>>>
>>> And it would be helpful to have some concrete suggestions for improving
>>> the
>>> API.
>>>
>>> Best Regards,
>>> Chris
>>>
>>>
>>
>>