[webrtc-pc] Proposing setCodecPreferences to deal with both send and recv codecs (#2939)

henbos has just created a new issue for https://github.com/w3c/webrtc-pc:

== Proposing setCodecPreferences to deal with both send and recv codecs ==
Based on prior discussions such as https://github.com/w3c/webrtc-pc/issues/2937#issuecomment-1942031155, here's a concrete proposal which I think fixes unidirectionality, requires no changes to JSEP (consistent with https://github.com/w3c/webrtc-pc/issues/2938#issuecomment-1942116384) and avoids backwards compat issues introduced by #2926.

*How did we get here?*

JSEP has this note (https://rtcweb-wg.github.io/jsep/#rfc.section.4.2.6):
> Note that setCodecPreferences does not **directly** affect which codec the implementation decides to send. It only affects which codecs the implementation indicates that it prefers to receive, via the offer or answer.

We took this to mean "SCP should only take _receive codecs_ as input", something which JSEP neither says or IMO imply. The note only talks about the *direct* effect; there's also the *indirect* effect that the preferences have.

*Excluding send codecs was a mistake.*

The preferences are relevant even for sendonly transceivers, since **we must have something to offer** (https://rtcweb-wg.github.io/jsep/#rfc.section.5.2.1):
> If codec preferences have been set for the associated transceiver, media formats MUST be generated in the corresponding order, and **MUST exclude any codecs not present in the codec preferences**.

This does not mean that preferences are irrelevant for the sending use case: **the receiver not having called `setCodecPreferences` is a valid use case**, in which case the sender's preferences are the ones we use (https://rtcweb-wg.github.io/jsep/#rfc.section.5.3.1):
> - If codec preferences have been set for the associated transceiver, media formats MUST be generated in the corresponding order, regardless of what was offered, and MUST exclude any codecs not present in the codec preferences.
> - **Otherwise,** the media formats on the m= line MUST be generated in the **same order as those offered** in the current remote description, excluding any currently unsupported formats. Any currently available media formats that are not present in the current remote description MUST be added after all existing formats.

We could simply say we don't care about this use case, but that appears to have some downsides when it comes to unidirectional codecs.

**Proposal**

"`setCodecPreferences` accepts both send codecs and receive codecs as input."

The actual preferences you end up using depend on the [createOffer filtering](https://w3c.github.io/webrtc-pc/#dfn-final-steps-to-create-an-offer) we already apply based on `transceiver.direction`. This makes it valid for...
- A sendonly transceiver to offer sendonly codecs and sendrecv codecs.
- A sendrecv transceiver to offer sendrecv codecs, but not sendonly or recvonly codecs.
- A recvonly transceiver to offer recvonly codecs and sendrecv codecs.

*To be discussed*

Note that it's possible for the filtering to end up with an empty codec list. There are multiple ways we could deal with this, of which I have no strong preference. Examples:
1. SCP and direction setting could sanity check and throw if we end up with an empty set.
2. createOffer could throw.
3. We don't throw, but if we have nothing to offer we reject the m= section.
4. If we end up with empty set we offer one of the mandated to support sendrecv codecs as a "backup" (VP8 and H264).


Please view or discuss this issue at https://github.com/w3c/webrtc-pc/issues/2939 using your GitHub account


-- 
Sent via github-notify-ml as configured in https://github.com/w3c/github-notify-ml-config

Received on Wednesday, 14 February 2024 09:36:10 UTC