Re: [webrtc-pc] scaleResolutionDownBy validation steps are inconsistent, browser interop issues (#2873)

> If so, what should the following produce? [deleting some but not all encodings' scaleResolutionDownBy]

In the addTransceiver case, 4:2:1 is only applied if none of the encodings have a scaleResolutionDownBy value. So I don't think we should apply 4:2:1 logic if there still exists an encoding (encodings[2]) that have a scaleResolutionDownBy value.

We'd still need to agree on what to do if we're trying to apply an undefined scale... not sure if this should throw, apply 1, or apply whatever was the scale previously. In practise I think apps either apply all or none of the scales, but throwing an exception has backwards compat risks that we may want to avoid. I'd check what browsers do today if we can find a pattern.

> ```
> const params = sender.getParameters();
> for (encoding of params.encodings) {
>   delete encoding.scaleResolutionDownBy;
> }
> await sender.setParameters(params)
> ```
> Why would anyone...

The pattern I see in practise is getting the parameters, then for a subset of the parameters (e.g. scaleResolutionDownBy) overwrite it to be what the app's current settings are and then setting the resulting parameters.

The issue for the app doing this pattern boils down to the app thinking "the current settings are scaleResolutionDownBy is undefined". So it setParameters() with scaleResolutionDownBy not having a value, as per current setting.
- In Chrome, this works because getParameters returns undefined and setParameters with undefined results in 4:2:1.
- In Firefox, this does not work because getParameters returns 4:2:1 and setParameters with undefined results in 1:1:1.
- In Safari, this doubly does not work because getParameters is 1:1:1 by default, but setParameters with undefined is also 1:1:1.

You could argue that the app should update its current settings to 4:2:1 and never set undefined scaleResolutionDownBy in setParameters, but today there is a preference towards not specifying scaleResolutionDownBy so we end up having to use "if browser..." gates to work around this.

> From my recollection of discussions on the existing behavior, we stopped short of applying magic defaults once parameters have been fiddled with, since they arguably stop being POLA at that point.

I recall the discussion being "after initial setup, scaleResolutionDownBy will have a value, so there's no need for the 4:2:1 algorithm anymore". This discussion did not anticipate the fact that:
- Apps having a current setting which distinguishes between undefined and 4:2:1.
- Some browsers return undefined in getParamers rather than 4:2:1 if the last value set was undefined.

-- 
GitHub Notification of comment by henbos
Please view or discuss this issue at https://github.com/w3c/webrtc-pc/issues/2873#issuecomment-1556713831 using your GitHub account


-- 
Sent via github-notify-ml as configured in https://github.com/w3c/github-notify-ml-config

Received on Monday, 22 May 2023 07:51:15 UTC