Re: Determining Output Latency

Actually, I think there are two different issues here.  One of them is the
latency measurement, which is useful to have for many reasons; the other is
around synchronizing audio events and other events - including MIDI, which
may have quite high-precision time stamps, but (obviously) in a different
time context.  To synchronize those events, regardless of how an
application defines its "master clock", in order to do proper
synchronization (e.g. "schedule sending this MIDI message and starting this
audio buffer at the same time"), as you said I think we'll need some form
of "at this audio time, it was/is this system time."  Or perhaps there's
some other way to do this - but I'm now hitting a wall in how to precisely
synchronize events in both worlds.


On Tue, Jan 22, 2013 at 4:56 PM, Srikumar Karaikudi Subramanian <
srikumarks@gmail.com> wrote:

> I think we need to know the *absolute* system time to which currentTime
> corresponds, *including* any latency. Here's why -
>
> An application that needs to visualize audio events that does its drawing
> in a requestAnimationFrame() callback needs to know the relationship
> between AudioContext.currentTime and the time stamp passed to the rAF
> callback. We do not have a way to determine the system time at which the
> audio clock started and the system clock's rate may be different from the
> audio clock's rate on some devices anyway. So this relationship needs to be
> estimated using a linear model, at least.
>
> The absence of a known relationship in this context can be compensated for
> if the audio system uses a small enough buffer size. Given an audio
> sampling rate of 44100Hz and a display refresh rate of 60Hz, we expect 735
> samples to pass during a visual frame. So if the audio buffer size is <=
> 256 samples, the drawing code can get away with using currentTime as its
> draw time, perhaps shifting it by one buffer duration into the future. If
> the buffer size is any larger, you'd get serious temporal aliasing effects
> if you use currentTime for this purpose.[1] A buffer size of 512 samples is
> therefore not good enough for this purpose and low power devices may need
> to use even longer buffers like 2048 to serve continuous audio.
>
> Only providing a "latency" value would not solve this "visual sampling"
> problem, and the original T+L problem presented in this thread would exist
> even if the client is using a very small buffer size. Both these problems
> can be solved if we know the absolute system time at which samples time
> stamped with currentTime will go out.
>
> Thoughts?
>
> Best,
> -Kumar
>
> [1] You'd still get some temporal jitter with 256, but it may be tolerable.
>
> On 17 Jan, 2013, at 1:51 AM, Chris Rogers <crogers@google.com> wrote:
>
>
>
> On Wed, Jan 16, 2013 at 12:16 PM, Joseph Berkovitz <joe@noteflight.com>wrote:
>
>> Thanks, Chris. This is certainly one way to handily solve the problem. I
>> also wonder whether an offset, rather than an absolute time, might be
>> cleaner. I don't have a strong preference.
>>
>
> The offset, I think, is what I'm suggesting with the
> AudioContext.presentationLatency attribute.  It would be in seconds
> (hopefully small in most cases :)
>
> Chris
>
>
>>
>> I neglected to give a use case but for completeness, here it is. If one
>> needs to display a visual cursor in relationship to some onscreen
>> representation of an audio timeline (e.g. a cursor on top of music notation
>> or DAW clips) then knowing the performance.now() coordinates for what is
>> coming out of the speakers is essential. Otherwise the display tells a lie.
>>
>> (This assumes that there is essentially zero latency in the visual
>> display but, that seems reasonable.)
>>
>> …Joe
>>
>> On Jan 16, 2013, at 2:05 PM, Chris Wilson <cwilso@google.com> wrote:
>>
>> Chris and I were talking about just this problem the other day, as I was
>> exploring synchronizing Web Audio events with other events (notably, of
>> course, MIDI) that are on the performance.now() clock.  After writing the
>> Web Audio scheduling article for HTML5Rocks (
>> http://www.html5rocks.com/en/tutorials/audio/scheduling/), I realized
>> that it's problematic to incorporate scheduling other real-time events
>> (even knowing precisely "what time it is" from the drawing function)
>> without a better understanding of the latency.
>>
>> The idea we reached (I think Chris proposed it, but I can't honestly
>> remember) was to have a performance.now()-reference clock time on
>> AudioContext that would tell you when the AudioContext.currentTime was
>> taken (or when that time will occur, if it's in the future); that would
>> allow you to synchronize the two clocks.  The more I've thought about it,
>> the more I quite like this approach - having something like
>> AudioContext.currentSystemTime in window.performance.now()-reference.
>>
>> On Wed, Jan 16, 2013 at 9:51 AM, Joseph Berkovitz <joe@noteflight.com>wrote:
>>
>>> Hi Chris,
>>>
>>> It's become apparent that on some devices and Web Audio implementations,
>>> an AudioContext's currentTime reports a time that is somewhat ahead of the
>>> time of the actual audio signal emerging from the device, by a fixed
>>> amount.  To be more specific, if a sound is scheduled (even very far in
>>> advance) to be played at time T, the sound will actually be played when
>>> AudioContext.currentTime = T + L where L is a fixed number which for the
>>> purposes of this email I'll call "output latency".
>>>
>>> I think the only reason this hasn't been noticed before is that until
>>> recently, the output latency on the implementations that I've been exposed
>>> to has been too small to notice. But in some implementations it can be
>>> substantial and noticeable.
>>>
>>> When this occurs, is this 1) a problem with the implementation of the
>>> spec, or 2) an anticipated phenomenon that may vary from one implementation
>>> to another?
>>>
>>> If the answer is 1), then at a minimum the spec needs to clarify the
>>> meaning of context.currentTime with respect to physical audio playback so
>>> that implementors realize they must add L back into the reported value of
>>> currentTime to make it correct.  But if the answer is 2), then we have a
>>> different problem: there doesn't appear to be any way to interrogate the
>>> API to determine the value of L on any particular platform.
>>>
>>> Can you or others on the list provide any guidance on this point? Should
>>> I file a bug and, if so, what for?
>>>
>>> Best,
>>>
>>> ... .  .    .       Joe
>>>
>>> *Joe Berkovitz*
>>> President
>>>
>>> *Noteflight LLC*
>>> Boston, Mass.phone: +1 978 314 6271
>>>       www.noteflight.com
>>> "Your music, everywhere"
>>>
>>>
>>
>>         ... .  .    .       Joe
>>
>> *Joe Berkovitz*
>> President
>>
>> *Noteflight LLC*
>> Boston, Mass. phone: +1 978 314 6271
>>       www.noteflight.com
>> "Your music, everywhere"
>>
>>
>
>