[webrtc-pc] Clarify the definition of "playout" for `RTCRtpContributingSource`. (#2172)

chxg has just created a new issue for https://github.com/w3c/webrtc-pc:

== Clarify the definition of "playout" for `RTCRtpContributingSource`. ==
This is related to [WebRTC Issue 10545](https://bugs.chromium.org/p/webrtc/issues/detail?id=10545).

There seem to be some confusion around the exact definition of "playout" for [RTCRtpContributingSource](https://w3c.github.io/webrtc-pc/#dom-rtcrtpcontributingsource):

1. After receiving a packet. Before it's added to the jitter buffer.

This interpretation is supported by:

> The browser MUST keep information from RTP packets _received_ in the previous 10 seconds.

2. After a packet is removed from the jitter buffer. Before the samples/frames are delivered to `MediaStreamTrack`.

This interpretation is supported by:

> When the first audio or video frame contained in an RTP packet is _delivered_ to the RTCRtpReceiver's MediaStreamTrack for playout...

3. Exactly when the packet payload is physically played out on the monitor/speaker.

This interpretation is supported by:

> The timestamp of type DOMHighResTimeStamp [HIGHRES-TIME], indicating the most recent time of _playout of media_ that arrived in an RTP packet originating from this source.

WebRTC currently implements (1). It updates `RTCRtpContributingSource` when a packet is received from the network. This is problematic. There are, for example, no guarantees that received packets are ever going to be played out (e.g. audio having already done concealment beyond the RTP timestamp of the packet payload). Packet reordering may also cause packets with lower sequence numbers to overwrite the information of packets with a higher sequence number.

I believe that (3) would be ideal in terms of usefulness, and ideal in terms of how a normal person would interpret the spec, but it's unfortunately not implementable in practice. An implementation of the spec will rarely have access to frame-perfect `onPhysicallyPlayedOut()` callbacks.

This leaves us with (2), which should be both implementable and useful.

Please view or discuss this issue at https://github.com/w3c/webrtc-pc/issues/2172 using your GitHub account

Received on Monday, 15 April 2019 14:00:17 UTC