Re: [webrtc-pc] setCodecPreferences vs unidirectional codecs (#2888)

I support making setCodecPreferences only control receiver side preferences, which shouldn't be controversial considering JSEP says they are receiver preferences. More motivation based on the JSEP rule about order below.

> "Any currently available media formats that are not present in the current remote description MUST be added after all existing formats"

This JSEP quote has implications. If I want to send H264 but receive VP8 I can do this:
1. No need to signal H264 in the offer since I prefer to _receive_ VP8, so `setCodecPreferences(VP8)`. But now I depend on the other endpoint adding H264. It can either add H264 and keep VP8 as a backup, but if so JSEP says it should do "VP8, H264" which implies it prefers VP8 even though it wants to prefer H264. It could either break this JSEP rule and do "H264, VP8" anyway (which would work but it seems strange to depend on endpoints breaking JSEP rules as the official way to do something) or it could pretend not to support VP8 and remove it, such that it can respond only with "H264". The latter option would work, but now we've prevented the other endpoint from dynamically switching to VP8, which might not be what we want to do either.
2. Another option is to offer `setCodecPreferences(VP8, H264)` in order to allow the other endpoint to make H264 first in the response, while still correctly signaling that we prefer to receive VP8 before H264. And this works today where all codecs have sendrecv capabilities, but how would this work if H264 was instead a sendonly codec that we don't know how to receive? What does it mean to prefer to receive something we don't know how to decode? Using the API this way seems to break down in a world of truly unidirectional codec capabilities.

Based on 2) breaking down, I think 1) is the only way to go, i.e. SDP says this is receiver preferences so let's treat it as such. Send codecs are irrelevant and don't need to be on the list.

So going with 1), how would we deal with a JSEP-respecting endpoint that responds "VP8, H264" instead of "H264, VP8"? I think the answer is @Orphis' setParameters([codec](https://w3c.github.io/webrtc-extensions/#dom-rtcrtpencodingparameters-codec):H264) which allows sending a codec regardless of its placement on the list, as long as it was negotiated.

> Open question: what does setCodecPreferences do for a sendonly m-line?

Not much, it sounds like. But what does JSEP say, is it allowed to have an empty list of codecs in the SDP or are we forced to add dummy codecs? The current API rule that says "treat empty list as using browser defaults" seems to not work here... backwards-compat issues incoming?

-- 
GitHub Notification of comment by henbos
Please view or discuss this issue at https://github.com/w3c/webrtc-pc/issues/2888#issuecomment-1838256233 using your GitHub account


-- 
Sent via github-notify-ml as configured in https://github.com/w3c/github-notify-ml-config

Received on Monday, 4 December 2023 10:30:11 UTC