[webrtc-pc] scaleResolutionDownBy validation steps are inconsistent, browser interop issues (#2873)

henbos has just created a new issue for https://github.com/w3c/webrtc-pc:

== scaleResolutionDownBy validation steps are inconsistent, browser interop issues ==
The validation steps are different for which scaling factors to use if scaleResolutionDownBy is not specified. When creating a transceiver we do 4:2:1 but when setting parameters we do 1:1:1.
- In [addTransceiver sendEncodings validation steps](https://w3c.github.io/webrtc-pc/#dfn-addtransceiver-sendencodings-validation-steps) we say: _If kind is "video" and none of the encodings contain a scaleResolutionDownBy member, then for each encoding, add a scaleResolutionDownBy member with the value 2^(length of sendEncodings - encoding index - 1). This results in smaller-to-larger resolutions where the last encoding has no scaling applied to it, e.g. 4:2:1 if the length is 3._
- In [Set the session description](https://w3c.github.io/webrtc-pc/#set-the-session-description), where remote SDP is processed and encodings are being configured for a "suitable transceiver", we say: _If proposedSendEncodings is non-empty, set each encoding's [scaleResolutionDownBy](https://w3c.github.io/webrtc-pc/#dom-rtcrtpencodingparameters-scaleresolutiondownby) to 2^(length of proposedSendEncodings - encoding index - 1)._
- In [setParameters](https://w3c.github.io/webrtc-pc/#dom-rtcrtpsender-setparameters) we say: _If transceiver kind is "video", then for each encoding in encodings that doesn't contain a scaleResolutionDownBy member, add a scaleResolutionDownBy member with the value 1.0._

What the spec describes (1:1:1) seems undesirable, can we be consistent and always do 4:2:1?

This inconsistency is causing different issues on different browsers, we need to align and I would prefer if we all align on 4:2:1 to avoid weird edge cases.

What happens today:
- In Chrome, clearing scaleResolutionDownBy in setParameters() triggers 4:2:1 but on Safari and Firefox it triggers 1:1:1. [Repro that starts in 4:2:1](https://orphis.github.io/webrtc-sandbox/simulcast-playground.html?resolution=HD&captureFramerate=30&codec=VP8&contentHint=&encodings.0.rid=&encodings.0.active=true&encodings.0.priority=low&encodings.0.maxBitrate=&encodings.0.scaleResolutionDownBy=4&encodings.0.maxFramerate=&encodings.0.scalabilityMode=&encodings.1.rid=1&encodings.1.active=true&encodings.1.priority=low&encodings.1.maxBitrate=&encodings.1.scaleResolutionDownBy=2&encodings.1.maxFramerate=&encodings.1.scalabilityMode=&encodings.2.rid=2&encodings.2.active=true&encodings.2.priority=low&encodings.2.maxBitrate=&encodings.2.scaleResolutionDownBy=1&encodings.2.maxFramerate=&encodings.2.scalabilityMode=&degradationPreference=&splitLayer=split) but you can trigger this by manually clearing scaleResolutionDownBy and pressing "Set Parameters" button. Chrome's behavior goes against the spec but I believe makes more sense.
- In Safari, creating a transceiver without specifying scaleResolutionDownBy triggers 1:1:1 ([repro](https://orphis.github.io/webrtc-sandbox/simulcast-playground.html?resolution=HD&captureFramerate=30&codec=VP8&contentHint=&encodings.0.rid=&encodings.0.active=true&encodings.0.priority=low&encodings.0.maxBitrate=&encodings.0.scaleResolutionDownBy=&encodings.0.maxFramerate=&encodings.0.scalabilityMode=&encodings.1.rid=1&encodings.1.active=true&encodings.1.priority=low&encodings.1.maxBitrate=&encodings.1.scaleResolutionDownBy=&encodings.1.maxFramerate=&encodings.1.scalabilityMode=&encodings.2.rid=2&encodings.2.active=true&encodings.2.priority=low&encodings.2.maxBitrate=&encodings.2.scaleResolutionDownBy=&encodings.2.maxFramerate=&encodings.2.scalabilityMode=&degradationPreference=&splitLayer=split)). This goes against the spec (at creation time spec says to do 4:2:1).
- In Firefox, creating a transceiver without specifying scaleResolutionDownBy triggers 4:2:1 ([repro](https://orphis.github.io/webrtc-sandbox/simulcast-playground.html?resolution=HD&captureFramerate=30&codec=VP8&contentHint=&encodings.0.rid=&encodings.0.active=true&encodings.0.priority=low&encodings.0.maxBitrate=&encodings.0.scaleResolutionDownBy=&encodings.0.maxFramerate=&encodings.0.scalabilityMode=&encodings.1.rid=1&encodings.1.active=true&encodings.1.priority=low&encodings.1.maxBitrate=&encodings.1.scaleResolutionDownBy=&encodings.1.maxFramerate=&encodings.1.scalabilityMode=&encodings.2.rid=2&encodings.2.active=true&encodings.2.priority=low&encodings.2.maxBitrate=&encodings.2.scaleResolutionDownBy=&encodings.2.maxFramerate=&encodings.2.scalabilityMode=&degradationPreference=&splitLayer=split)). This is what the spec says, but if you press "Set Parameters" without touching scaleResolutionDownBy (meaning it is still undefined) you get 1:1:1. You could argue this is an issue with the test page rather than the browser since getParameters() does return 4:2:1 and a "get+setParameters" dance would have applied 4:2:1 correctly, but the issue remains that the parameters changed behind our backs between addTransciever and getParameters. The last thing that was set was not what you obtained in getParameters.

Please view or discuss this issue at https://github.com/w3c/webrtc-pc/issues/2873 using your GitHub account


-- 
Sent via github-notify-ml as configured in https://github.com/w3c/github-notify-ml-config

Received on Monday, 15 May 2023 15:30:13 UTC