Re: [EXTERNAL] Re: A proposal for RTP header extension control

When we implemented RTP header extension selection in ORTC, we found a number of "gotchas", relating to ID conflicts, some of which were quite subtle. For example, when senders/receivers are bundled, it is possible for the IDs on multiple m-lines to conflict, even though the IDs are unique within an m-line.

The most serious issue resulted from confusion with the MID header extension.  In order to route RTP packets, it is necessary to know what ID corresponds to the MID extension. Assigning multiple IDs to MID or disabling the MID header extension on the RtpReceiver after it had been negotiated can resulting in RTP packets being mis-routed or worse (e.g. crashes).

It might be hard to articulate all the opportunities for mischief ahead of time, so I like the idea of foreclosing unnecessary functionality (e.g. only making it possible to disable negotiated extensions, no changing IDs, etc.).
________________________________
From: Harald Alvestrand <harald@alvestrand.no>
Sent: Tuesday, January 14, 2020 7:57 AM
To: public-webrtc@w3.org <public-webrtc@w3.org>
Subject: [EXTERNAL] Re: A proposal for RTP header extension control



On 1/14/20 4:15 PM, Henrik Boström wrote:
These APIs are on a per-transceiver basis. Are RTP header extensions negotiated on a per-transceiver basis, or is it on a "for all m= sections in the offer"-, "the offerer tagged m= section" or "per transport"-basis?
If they're negotiated per m= section always then usage seems clear, otherwise I think we need to figure out what to do. For example, maybe we'd be forced to negotiate an RTP header extension for transceiver0 because transceiver1 needs it if RTP header extensions are negotiated for the entire offer, in which case we'd need to make transceiver0.[sender/receiver].getParameters() default to "inactive" values.

https://tools.ietf.org/html/draft-ietf-mmusic-sdp-mux-attributes-16#section-5.13<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftools.ietf.org%2Fhtml%2Fdraft-ietf-mmusic-sdp-mux-attributes-16%23section-5.13&data=02%7C01%7CBernard.Aboba%40microsoft.com%7C440a900114d04692dc0008d7990aa99f%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637146143390646516&sdata=W17wK0qOc%2BpZcZV1xpXhE53txy7CHFGuIRLX6sfs18Q%3D&reserved=0> says that they're in category SPECIAL - one must understand the extension in order to figure out what makes sense.

But numbers used on a single transport can't collide, so in that sense, they're negotiated for the whole transport - the number allocator has to be per-transport.

Just-generated example:

Audio section:

a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level
a=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time<https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.webrtc.org%2Fexperiments%2Frtp-hdrext%2Fabs-send-time&data=02%7C01%7CBernard.Aboba%40microsoft.com%7C440a900114d04692dc0008d7990aa99f%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637146143390656482&sdata=spwgXobPs6pYQ4NBrUA%2F63BraBxQBK6vU5mWDQeFjE4%3D&reserved=0>
a=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01<https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.ietf.org%2Fid%2Fdraft-holmer-rmcat-transport-wide-cc-extensions-01&data=02%7C01%7CBernard.Aboba%40microsoft.com%7C440a900114d04692dc0008d7990aa99f%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637146143390656482&sdata=BLbWnmTMWCr%2FZzTJ8Rf1qDtF7ku7XqwD0OPMHAPTCK0%3D&reserved=0>
a=extmap:4 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:5 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id
a=extmap:6 urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id

Video section:

a=extmap:14 urn:ietf:params:rtp-hdrext:toffset
a=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time<https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.webrtc.org%2Fexperiments%2Frtp-hdrext%2Fabs-send-time&data=02%7C01%7CBernard.Aboba%40microsoft.com%7C440a900114d04692dc0008d7990aa99f%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637146143390666436&sdata=oKpt6nkpx6NS5EUrQTeiYbwyNKhB9Ey7VppRY7QZpVU%3D&reserved=0>
a=extmap:13 urn:3gpp:video-orientation
a=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01<https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.ietf.org%2Fid%2Fdraft-holmer-rmcat-transport-wide-cc-extensions-01&data=02%7C01%7CBernard.Aboba%40microsoft.com%7C440a900114d04692dc0008d7990aa99f%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637146143390666436&sdata=pELEDh8V6JxHSfpJ1Hy8fnhnASHhaOqtNQ62fmyT82U%3D&reserved=0>
a=extmap:12 http://www.webrtc.org/experiments/rtp-hdrext/playout-delay<https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.webrtc.org%2Fexperiments%2Frtp-hdrext%2Fplayout-delay&data=02%7C01%7CBernard.Aboba%40microsoft.com%7C440a900114d04692dc0008d7990aa99f%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637146143390666436&sdata=3wC1J2VhnIKXti5CdxJxgx1a%2F%2Fa1h%2Bf8zL1%2Bz91IJFo%3D&reserved=0>
a=extmap:11 http://www.webrtc.org/experiments/rtp-hdrext/video-content-type<https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.webrtc.org%2Fexperiments%2Frtp-hdrext%2Fvideo-content-type&data=02%7C01%7CBernard.Aboba%40microsoft.com%7C440a900114d04692dc0008d7990aa99f%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637146143390676393&sdata=E21iLLMN0dP4zKRbESrRtllJIC520FhZvigk7I%2FOO3E%3D&reserved=0>
a=extmap:7 http://www.webrtc.org/experiments/rtp-hdrext/video-timing<https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.webrtc.org%2Fexperiments%2Frtp-hdrext%2Fvideo-timing&data=02%7C01%7CBernard.Aboba%40microsoft.com%7C440a900114d04692dc0008d7990aa99f%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637146143390676393&sdata=TawiW4scigFAuyPXIiu85zRpi67%2B561Mv9b6KTnTAcI%3D&reserved=0>
a=extmap:8 http://tools.ietf.org/html/draft-ietf-avtext-framemarking-07<https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Ftools.ietf.org%2Fhtml%2Fdraft-ietf-avtext-framemarking-07&data=02%7C01%7CBernard.Aboba%40microsoft.com%7C440a900114d04692dc0008d7990aa99f%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637146143390686346&sdata=iFh4eFnCIhgXvQBANxedXI49XeJhXnMJ0xcDIcuZq9E%3D&reserved=0>
a=extmap:9 http://www.webrtc.org/experiments/rtp-hdrext/color-space<https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.webrtc.org%2Fexperiments%2Frtp-hdrext%2Fcolor-space&data=02%7C01%7CBernard.Aboba%40microsoft.com%7C440a900114d04692dc0008d7990aa99f%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637146143390686346&sdata=lSsWez9gTo5KhjDZm8b6Z%2BPC4icYyFJ8oPtzqDeubb4%3D&reserved=0>
a=extmap:4 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:5 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id
a=extmap:6 urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id

As you can see, the same extension in the two media sections uses the same number.






On Mon, Jan 13, 2020 at 1:33 PM Harald Alvestrand <harald@alvestrand.no<mailto:harald@alvestrand.no>> wrote:
Den 13.01.2020 11:11, skrev Youenn Fablet:
> I like the idea of exposing more things through API.
> I am curious about the reasons behind surfacing the API at transceiver
> instead of sender/receiver level.


My rule of thumb is that APIs that don't depend on SDP negotiation
belong to sender/receiver, while APIs that chiefly exist to influence
SDP negotiation belong to transceiver.

I still have a long term hope that we'll get to a state where we can
have sender/receiver objects whose configuration can be managed without
reference to SDP (but for the moment, I'm just trying to not make that
goal be further away).

>
>> On 13 Jan 2020, at 10:51, Harald Alvestrand <harald@alvestrand.no<mailto:harald@alvestrand.no>
>> <mailto:harald@alvestrand.no<mailto:harald@alvestrand.no>>> wrote:
>>
>> For those who like Google docs, there's a document with public
>> comments enabled here:
>>
>>
>> https://docs.google.com/document/d/1y1hTsMeav5ijPvoqu1R6U4YC564i1QzgkMeIqWhgiis/edit#<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.google.com%2Fdocument%2Fd%2F1y1hTsMeav5ijPvoqu1R6U4YC564i1QzgkMeIqWhgiis%2Fedit%23&data=02%7C01%7CBernard.Aboba%40microsoft.com%7C440a900114d04692dc0008d7990aa99f%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637146143390686346&sdata=aAg%2FBvW4leri%2FXLAJjOAISwIl83nHQ%2BUzfoZtOhmQ4c%3D&reserved=0>
>>
>>
>> On 1/13/20 2:11 AM, Harald Alvestrand wrote:
>>>
>>> This is a proposal for an extension to the WebRTC-PC specification in
>>> order to allow control of which RTP headers are negotiated and used.
>>>
>>> If time allows, I'll present this at the Tuesday Virtual Interim meeting.
>>>
>>> If the group finds the proposal reasonable, I'll prepare a pull
>>> request against the WebRTC extensions document
>>> (https://github.com/w3c/webrtc-extensions<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fw3c%2Fwebrtc-extensions&data=02%7C01%7CBernard.Aboba%40microsoft.com%7C440a900114d04692dc0008d7990aa99f%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637146143390696306&sdata=o0YUzPrfWNoo1j6G6VnUtLjqf0ORVVgtWN2EP%2BmoJoU%3D&reserved=0>)
>>>
>>> This proposal is NOT meant to be merged with the WEBRTC-PC document
>>> that we're currently trying to get to Proposed Recommendation.
>>>
>>> *Harald*
>>>
>>> **
>>>
>>>
>>>   *RTP header extension control*
>>>
>>> *
>>> The standard RTP header extension mechanism is described in RFC 8285
>>> <https://tools.ietf.org/html/rfc8285<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftools.ietf.org%2Fhtml%2Frfc8285&data=02%7C01%7CBernard.Aboba%40microsoft.com%7C440a900114d04692dc0008d7990aa99f%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637146143390696306&sdata=10d5wp%2BJFrhl8GVBh%2BrJlEM39x2uRCEDWsFXoovYT%2F0%3D&reserved=0>>.
>>>
>>> We have two issues with RTP headers:
>>>
>>>  *
>>>     There are so many of them, and so many supported in browsers,
>>>     that we’re running into issues in SDP parsers with just offering
>>>     every RTP header extension that there is support for.
>>>  *
>>>     There are extensions that we don’t want enabled by default, or
>>>     not enabled all the time. Sometimes they are alternates and we
>>>     want to pick one.
>>>
>>>
>>> An extension API is needed for this. It needs to:
>>>
>>>  *
>>>     Be able to figure out what extensions (by name) the platform supports
>>>  *
>>>     Enable the negotiation of these extensions on initial SDP negotiation
>>>  *
>>>     Enable or disable these extensions on a specific RTP stream.
>>>
>>>
>>>     Proposal
>>>
>>> This proposal seems to satisfy the requirements above.
>>>
>>> It consists of two new functions on the RTCRtpTransceiver, to control
>>> the SDP negotiation, and one new attribute in
>>> RTCRtpHeaderExtensionParameters, which allows the enabling or
>>> disabling of an extension using setParameters().
>>>
>>>
>>>       Controlling the SDP negotiation
>>>
>>> IDL:
>>>
>>> partial dictionary RTCRtpHeaderExtensionParameters {
>>>        RTCRtpTransceiverDirection direction = “sendrecv”;
>>>             // Sendonly, recvonly, sendrecv, inactive, stopped
>>> }
>>> // This also redefines RTCRtpHeaderExtensionList
>>> dictionary RTCRtpHeaderExtensionCapabilityWithDirection
>>>   : RTCRtpHeaderExtensionCapability {
>>>      RTCRtpTransceiverDirection direction = “sendrecv”;
>>> }
>>>
>>> partial interface RTCRtpTransceiver {
>>>    void setOfferedRtpHeaderExtensions(
>>>     RTCRtpHeaderExtensionCapabilityWithDirection
>>> headerExtensionsToOffer);
>>>    readonly attribute RTCRtpHeaderExtensionList headerExtensionsAccepted;
>>> }
>>>
>>> The “headerExtensionsToOffer” argument will control what is offered
>>> in the next negotiation.
>>> The following meanings attach to the “direction” attribute:
>>>
>>>  *
>>>     Sendonly, recvonly, sendrecv, inactive: These will be signalled
>>>     in the SDP. If the attribute is “sendonly” or “sendrecv”, the
>>>     attribute will be used by the attached sender on this transceiver.
>>>  *
>>>     stopped : Will NOT be signalled in the SDP.
>>>
>>>
>>> The URIs listed in the argument to “setOffered” MUST be a subset of
>>> the union of the capabilities listed by sender.getCapabilities /
>>> receiver.getCapabilities. If an unrecognized URI is present,
>>> “typeError” will be thrown.
>>>
>>> If any RTP header extension that the platform regards as mandatory to
>>> send is set to “recvonly”, “inactive” or “stopped”, or an RTP header
>>> extension that the platform regards as mandatory to receive is set to
>>> “sendonly”, “inactive” or “stopped”, the call will thrown a
>>> “InvalidModificationError”.
>>>
>>> The “accepted” attribute will be empty until an offer/answer has been
>>> completed.
>>>
>>>
>>>       Controlling the RTP extensions in use
>>>
>>>
>>> Reporting what’s in use is already handled by the
>>> RTCRtpHeaderExtensionParameterselement on the sender/receiver. This
>>> also gives the extension numbers, if needed.
>>>
>>> To figure out what’s available, one can use
>>> RTCRtpSender.getCapabilities() and look at the header extensions
>>> section; this returns a sequence<RTCRtpHeaderExtensionCapability>.
>>>
>>> To control what extensions are being sent, modify the “direction”
>>> attribute - setting it to “stopped” will prevent the RTP header
>>> extension from being sent.
>>> For this, one can use the usual getParameters/setParameters functions.
>>> Attempting to modify the receiver’s “direction”, or attempting to
>>> disable a mandatory parameter, will lead to rejection of the
>>> setParameters().
>>>
>>>
>>> *
>