Re: Reviewing the Web Audio API (from webrtc)

On Tue, Apr 10, 2012 at 1:21 PM, Randell Jesup <randell-ietf@jesup.org>wrote:

>  >On Thu, Mar 29, 2012 at 6:32 PM, Chris Rogers <crogers@google.com <crogers@google.com?Subject=Re%3A%20Reviewing%20the%20Web%20Audio%20API%20%28from%20webrtc%29&In-Reply-To=%253CCAOp6jLa4UShFqAeOqWHFj8w0VfMWOoMGM5kstmUDQqJEMGa%2BOA%40mail.gmail.com%253E&References=%253CCAOp6jLa4UShFqAeOqWHFj8w0VfMWOoMGM5kstmUDQqJEMGa%2BOA%40mail.gmail.com%253E>> wrote:
> >
> >> None of the built-in Web Audio processing algorithms have any appreciable
> >> latency which would perceptibly affect audio/video sync.
> >
> >
> >OK, but there are processing algorithms that necessarily have significant
> >latency, like this one:> http://people.mozilla.org/~roc/stream-demos/video-with-extra-track-and-effect.html <http://people.mozilla.org/%7Eroc/stream-demos/video-with-extra-track-and-effect.html>
>
> >
> >> We're talking about 3ms or less here.  In terms of irritation, network
> >> latency is of vastly more concern for WebRTC applications.
> >
> > That depends on the application. WebRTC APIs can be used for more than just
> > interactive chat. For example, an application could pull an audio and video
> > stream from some source, take a user's commentary in a stream from the
> > microphone, mix them with a ducking effect, and stream the resulting audio
> > and video out to a set of peers. The latency might be too high for
> > interaction, but just fine for a "live broadcast".
>
> Another example is an AGC, Dynamic Range Compressor, or Limiter using a look-ahead
> to avoid having to use extreme attack values.  This could induce 5, 10 or more ms
> of delay.
>
> Agreed, and the WebKit implementation of DynamicsCompressorNode in the Web
Audio API implements a small look-ahead (around 3ms) for exactly this
purpose.  However, it is very unlikely to affect audio/video sync in any
perceptible way.  The audio hardware latency is often much longer than this
(for some platforms can be dozens of ms).  Also, video frame presentation
latency can also become significant.  In any case, an implementation can
account for these latencies to the extent that the values are known (which
generally is the case for audio processing algorithms).

Cheers,
Chris