[webrtc-extensions] Negotiationless codec selection and per-encoding layer codec configuration (#126)

henbos has just created a new issue for https://github.com/w3c/webrtc-extensions:

== Negotiationless codec selection and per-encoding layer codec configuration ==
## Problem
With the introduction of scalabilityMode, we now have two competing APIs for configuring the encoder: setCodecPreferences (or SDP munging) and setParameters() with scalabilityMode. Issues:

1. Different codecs have different scalability modes and toggling "simulcast VP8 without SVC" and "non-simulcast VP9 using SVC" not only changes scalability, it also changes the number of active layers. Not being able to set all settings (=codec, scalability mode, active) in a single API call inevitably leads to temporarily sending the wrong thing. Its racy, causes additional key frames and not very ergonomic.

2. Some apps don't want to re-negotiate (e.g. fixed number of transceiver setups), besides, why do a round-trip to the server if you already know the server supports the desired codec?

3. An app may want to use different codec configurations for different layers e.g. performance vs bitrate tradeoffs.

## Proposal
Configure send codecs with setParameters(). Add "codec" to [RTCRtpEncodingParameters](https://w3c.github.io/webrtc-svc/#rtcrtpencodingparameters) of type [RTCRtpCodecCapability](https://w3c.github.io/webrtc-pc/#dom-rtcrtpcodeccapability) (same type that is used by getCapabilities and setCodecPreferences).

This means that the codec list order in the SDP will not dictate what codec we use for sending. What about backwards-compatibility? Don't worry, we still default to the first codec in the list, but if the codec value is ever changed, that's what we use going forward.

## Pseudo-PR

Update RTCRtpSender initialization steps:
1. Let sender have a [[PreferSelectedCodecs]] internal slot, initialized to false.
2. Let each encoding in sender's [[SendEncodings]] have a [[SelectedCodec]] internal slot representing the currently selected codec for this encoding, initialized to null.

Update the RTCRtpSender.setParameters() steps:
1. For each encoding in the argument to this function, let codec be the value of encodings[i].codec:
1.1. If codec is not one of the values in RTCRtpSender.getCapabilities(kind).codecs, throw an exception and abort these steps.
1.2. If transceiver.mid is not null and codec is not one of the values in [[SendCodecs]], throw an exception and abort these steps. _(If transceiver.mid is null we've not negotiated yet and allow optimistically selecting one.)_
1.3. If codec is not equal to the encoding's [[SelectedCodec]], set the encoding's [[SelectedCodec]] to codec and set [[PreferSelectedCodec]] to true. In parallel, reconfigure the this encoding's encoder to use codec.

Update the SDP negotiation steps, where [[SendCodecs]] is updated and after the number of [[SendEncodings]] have been established:
1. For each encoding in [[SendEncodings]]:
1.1. If [[PreferSelectedCodecs]] is false, or if the encoding's [[SelectedCodec]] is not in the list of [[SendCodecs]], set the encoding's [[SelectedCodec]] to the first element in [[SendCodecs]]. _(Handles default value and backwards-compat for apps relying on setCodecPreferences to change codec.)_
1.2. Reconfigure the encoding's encoder to use [[SelectedCodec]].


Please view or discuss this issue at https://github.com/w3c/webrtc-extensions/issues/126 using your GitHub account


-- 
Sent via github-notify-ml as configured in https://github.com/w3c/github-notify-ml-config

Received on Thursday, 1 December 2022 07:55:36 UTC