Re: [webrtc-pc] setCodecPreferences vs unidirectional codecs (#2888)

Trying to summarize the current state of discussion (from [this code change](https://webrtc-review.googlesource.com/c/src/+/328780?tab=comments)).

[JSEP](https://www.rfc-editor.org/rfc/rfc8829.html#name-setcodecpreferences) is pretty clear:
> setCodecPreferences does not directly affect which codec the implementation decides to send. It only affects which codecs the implementation indicates that it prefers to receive

So sCP (setCodecPreferences ) is not something you use to pick the send codec. 
This works just because send codec is dictated by receive preference in SDP semantics

sCP should not take send codecs into account which webrtc-pc does [here](https://w3c.github.io/webrtc-pc/#dom-rtcrtptransceiver-setcodecpreferences).

Answer may include codecs not offered, see https://www.rfc-editor.org/rfc/rfc8829.html#section-5.3.1
> "Any currently available media formats that are not present in the current remote description MUST be added after all existing formats"

No browsers do that, but [behave as intended](https://jsfiddle.net/fippo/gqasr6d1/1/) (or can be tricked?)

sCP should typically be done in ontrack, see updated [samples PR](https://github.com/webrtc/samples/pull/1640), e.g.
`e.transceiver.setCodecPreferences` instead of `addTransceiver` + `setCodecPreferences`. This often gets [nicer in terms of architecture](https://github.com/webrtc/samples/pull/1644)

This hopefully is a somewhat web-compatible change. Woes with unidirectional codecs were not discovered for years
It will still be surprising for developers...

The proposed fix is threefold:
1. Fix webrtc-pc by removing mentions of send codecs in setCodecPreferences. [preview here](https://github.com/w3c/webrtc-pc/compare/main...fippo:webrtc-pc:setcodecpreferences-recvonly?expand=1)
2. Clarify “[codecs match](https://w3c.github.io/webrtc-pc/#dfn-codec-match)” algorithm which says
> “If either (but not both) of first.sdpFmtpLine and second.sdpFmtpLine are missing, or if they both exist and first.sdpFmtpLine is different from second.sdpFmtpLine, return false.”
  * H264 [profile level-asymmetry-allowed](https://www.rfc-editor.org/rfc/rfc6184#section-8.1) needs to be taken into account
  * Inference of default values might need to be handled as well
3. drive-by: the spec is missing consideration for Comfort Noise (oddball, not a media codec, not a resiliency mechanism)

Open question: what does setCodecPreferences do for a sendonly m-line?
Current behavior: only negotiate receive codecs for symmetry


-- 
GitHub Notification of comment by fippo
Please view or discuss this issue at https://github.com/w3c/webrtc-pc/issues/2888#issuecomment-1838178048 using your GitHub account


-- 
Sent via github-notify-ml as configured in https://github.com/w3c/github-notify-ml-config

Received on Monday, 4 December 2023 09:44:33 UTC