[webrtc-pc] RTCRtpContributingSource.audioLevel not guaranteed to be in sync with audio playout

taylor-b has just created a new issue for https://github.com/w3c/webrtc-pc:

== RTCRtpContributingSource.audioLevel not guaranteed to be in sync with audio playout ==
My assumption is that this feature exists so that applications can show audio level UI indications for different participants of a call.

However, I don't see how this can be done in a robust manner, since the `RTCRtpContributingSource` objects are updated whenever a packet is received and not when audio is played out, with how the spec currently reads:

> Each time an RTP packet is received, the RTCRtpContributingSource objects are updated.

Consider these situations:

1. There is a noticeable delay between packets being received and audio playing out, due to poor network conditions, resulting in an audioLevel that's updated well in advance of audio playout; e.g., you see the volume indicator move before the speaker opens their mouth.
2. Traffic is bursty, resulting in the audio level jumping around when there's a burst of traffic, then remaining stagnant for a while.
3. Packets arrive out of order... and `timestamp` actually decreases?

How can these problems be mitigated? Could we change "Each time an RTP packet is received" to "each time a frame of media is delivered to the MediaStreamTrack" (or whatever the right terminology there is)?

Otherwise, what can an application do? Use `getStats` to figure out the playout delay, and then delay updating the audio level UI for that amount of time?

Please view or discuss this issue at https://github.com/w3c/webrtc-pc/issues/1085 using your GitHub account

Received on Thursday, 16 March 2017 20:27:13 UTC