W3C home > Mailing lists > Public > public-audio@w3.org > January to March 2012

Audio APIs Feedback

From: Samuel Thompson <sam@railgun3d.com>
Date: Thu, 2 Feb 2012 10:45:29 -0800
Message-ID: <CAMYBkFdmMwAgtt9AkjOB-9uHdcOHdou8F8CnutoLMjskzh3-ZA@mail.gmail.com>
To: public-audio@w3.org
Hi,

I just wanted to give some feedback on my experience working with the two
current audio APIs from the perspective of a console game engine programmer
making the transition to WebGL game development. Someone recommended I post
it here, so I hope this is the right place for this sort of thing and you
find my feedback useful.

I recently added audio support to our WebGL-based game engine, and since
we're trying to keep everything plugin-free, I wanted to avoid using Flash
and stick to HTML5 and the two more advanced Audio APIs; Chrome's Web Audio
API and Mozilla's Audio Data API. My requirements are pretty modest: I want
to play multiple samples at the same time and be able to adjust the gain
and pan on individual playbacks to simulate 3D positional sound. A stretch
goal would be more advanced sound modulation to simulate things like
long-distance low pass filtering, indoor reverb, or doppler pitch shifting.

It was pretty easy to get positional sound working with the Web Audio API,
and it looks like it will be fairly simple to add the more advanced effects
as well. To accomplish the same thing in the Audio Data API, as far as I
can tell, for each buffer that is currently playing I would need to
manually loop through the buffer in javascript, multiply the sample
information for each channel based on a panning factor, all while keeping
enough buffer to prevent pops (especially if we get a random GPU or garbage
collection stall), but not so much that real time modulation becomes
impossible (imagine a rocket flying by, we need to update the panning
information fairly quickly to simulate the positional effect of a moving
object). This may all be possible on older hardware, and there might even
be js libraries that handle this sort of thing smoothly (though I have yet
to find any), but either way it seems like a lot of resources wasted having
javascript do something that could be done at least natively if not through
hardware sound acceleration. It's entirely possible that I'm
overcomplicating it, so if there is a simpler or cheaper way to have
multichannel 3D positional sound using the Audio Data API, please let me
know!

At the moment my solution is to use the webkitAudioContext if it exists,
and if not, I actually just fall back to HTML5 audio elements and use the
volume controls on those for distance falloff. I just give up on panning or
any advanced effects in that case. As an aside, I am baffled that I can
change the playback rate of a sound with an HTML5 audio element but I can't
change the panning. Anyway, I totally understand the point of the low level
Audio Data API approach, and as a hobbyist electronic musician I am excited
about the possibilities such an approach allows, but as a professional game
programmer it seems much more frustrating than it needs to be.

Thanks,
Sam
Received on Sunday, 5 February 2012 20:12:31 GMT

This archive was generated by hypermail 2.2.0+W3C-0.50 : Sunday, 5 February 2012 20:12:32 GMT