W3C home > Mailing lists > Public > public-webrtc-logs@w3.org > April 2019

Re: [webrtc-pc] Add RTP timestamp to RTCRtpContributingSource (#2177)

From: Chen Xing via GitHub <sysbot+gh@w3.org>
Date: Fri, 26 Apr 2019 12:03:35 +0000
To: public-webrtc-logs@w3.org
Message-ID: <issue_comment.created-487033029-1556280214-sysbot+gh@w3.org>
I think that a per-frame ID (of some kind) would be very nice to have for `RTCRtpContributingSource`. For example: It would allow a caller to tell if two consecutive `getContributingSources()` calls returned the info about the exact same frame, or if they returned info about two different frames but `timestamp` didn't have enough resolution to tell the frames apart.

There are a bunch of challenges for the spec:

1. A single RTP packet may contain a payload that maps to multiple frame-deliveries to `MediaStreamTrack`.

    For example:
    - Audio packets store 20 ms of audio but WebRTC delivers 10 ms per `GetAudioFrameWithInfo()` call.

2. Multiple RTP packets may be involved in the decoding of a single frame.

    For example:
    - Video key frames.
    - Audio packets that are not aligned with audio outputs. Say we have a packet with audio for time period [1005, 1025) and another for [1025, 1045), but WebRTC is outputing [1020, 1030). It would then take audio samples from both packets.
    - Audio during acceleration. Similar to the case above.

3. Frames that are decoded from packets that don't follow the usual rules about RTP timestamps and sequence numbers.

    For example:
    - Transport-level forward error correction.
      - E.g. Flexfec.
      - We'll presumably use the recovered RTP timestamp.
    - In-band forward error correction.
      - E.g. Opus FEC where WebRTC can use the data from packet N+1 to decode a low-quality version of packet N.
      - Do we use the RTP timestamp for N+1 or the "would have been" RTP timestamp for N?
      - Probably most sense to compute and use the info for N?
      - Or maybe not record anything for this case at all?
    - Retransmission.
      - The retransmitted packet should have the same RTP timestamp as the original.

-- 
GitHub Notification of comment by chxg
Please view or discuss this issue at https://github.com/w3c/webrtc-pc/issues/2177#issuecomment-487033029 using your GitHub account
Received on Friday, 26 April 2019 12:03:40 UTC

This archive was generated by hypermail 2.4.0 : Saturday, 6 May 2023 21:19:47 UTC