Re: [webrtc-pc] Clarify the definition of "playout" for `RTCRtpContributingSource`. (#2172)

Thanks for the information!

So if I understand it correctly, then Firefox stores a `RTCRtpContributingSource`-like record at (1) but sets its timestamp to be an estimate of (2). Chrome is more naive in that it both records at (1) and timestamps for (1). Neither implementation tries to timestamp for (3) today. The exact lines of code for audio seem to be in [`Channel::OnReceivedPayloadData()`](https://dxr.mozilla.org/mozilla-central/rev/02b89c29412b6c1444fe32a4847e5261e2bb3d00/media/webrtc/trunk/webrtc/voice_engine/channel.cc#721) for Firefox and [`ChannelReceive::OnRtpPacket()`](https://cs.chromium.org/chromium/src/third_party/webrtc/audio/channel_receive.cc?l=594&rcl=852bffb6063aa36814c89a58390581e59643f1fb) for Chrome.

Would it make sense to improve the spec so that it better matches our two implementations?

My proposal would be to make the spec so that:

- Each `RTCRtpContributingSource` stores info about precisely one audio or video frame. This is already strongly hinted at by the existing spec. Aggregated stats that span multiple frames are more appropriately placed in [RTCStats](https://w3c.github.io/webrtc-pc/#rtcstats-dictionary).

- Each `RTCRtpContributingSource` is ideally recorded when the audio or video frame is delivered to RTCRtpReceiver's MediaStreamTrack for playout. I will make changes to WebRTC to move the recording of the info from `OnRtpPacket()` to [`ChannelReceive::GetAudioFrameWithInfo()`](https://cs.chromium.org/chromium/src/third_party/webrtc/audio/channel_receive.cc?l=316&rcl=852bffb6063aa36814c89a58390581e59643f1fb) for audio and [`VideoReceiveStream::OnFrame`](https://cs.chromium.org/chromium/src/third_party/webrtc/video/video_receive_stream.cc?l=487&rcl=852bffb6063aa36814c89a58390581e59643f1fb) for video.

- `RTCRtpContributingSource.timestamp` is re-defined as storing a timestamp of when the audio or video frame is delivered to RTCRtpReceiver's MediaStreamTrack for playout.

I believe that making the spec more precise would be helpful for when we later want to add more detailed breakdowns of per-frame timestamps.

-- 
GitHub Notification of comment by chxg
Please view or discuss this issue at https://github.com/w3c/webrtc-pc/issues/2172#issuecomment-485915658 using your GitHub account

Received on Tuesday, 23 April 2019 18:13:42 UTC