W3C home > Mailing lists > Public > public-webrtc@w3.org > March 2017

[webrtc-pc] RTCRtpContributingSource.audioLevel not guaranteed to be in sync with audio playout

From: Taylor Brandstetter via GitHub <sysbot+gh@w3.org>
Date: Thu, 16 Mar 2017 20:27:07 +0000
To: public-webrtc@w3.org
Message-ID: <issues.opened-214828228-1489696025-sysbot+gh@w3.org>
taylor-b has just created a new issue for https://github.com/w3c/webrtc-pc:

== RTCRtpContributingSource.audioLevel not guaranteed to be in sync with audio playout ==
My assumption is that this feature exists so that applications can show audio level UI indications for different participants of a call.

However, I don't see how this can be done in a robust manner, since the `RTCRtpContributingSource` objects are updated whenever a packet is received and not when audio is played out, with how the spec currently reads:

> Each time an RTP packet is received, the RTCRtpContributingSource objects are updated.

Consider these situations:

1. There is a noticeable delay between packets being received and audio playing out, due to poor network conditions, resulting in an audioLevel that's updated well in advance of audio playout; e.g., you see the volume indicator move before the speaker opens their mouth.
2. Traffic is bursty, resulting in the audio level jumping around when there's a burst of traffic, then remaining stagnant for a while.
3. Packets arrive out of order... and `timestamp` actually decreases?

How can these problems be mitigated? Could we change "Each time an RTP packet is received" to "each time a frame of media is delivered to the MediaStreamTrack" (or whatever the right terminology there is)?

Otherwise, what can an application do? Use `getStats` to figure out the playout delay, and then delay updating the audio level UI for that amount of time?

Please view or discuss this issue at https://github.com/w3c/webrtc-pc/issues/1085 using your GitHub account
Received on Thursday, 16 March 2017 20:27:13 UTC

This archive was generated by hypermail 2.3.1 : Monday, 23 October 2017 15:19:50 UTC