Constraints and Encoding Parameters

Before expressing an opinion relating to the consensus call, I thought it might be useful to have a bit of a discussion about this. 

In general, the principle behind RTCRtpEncodingParameters is to avoid duplicating functionality in gUM constraints.  

Since gUM already supports frameRate constraints (min, max, etc.) before adding maxFramerate to RTCRtpEncodingParameters, we should understand what scenarios require it. 

First off, there are some simulcast scenarios involving maximum framerates that do not require maxFramerate in RTCRtpEncodingParameters.  For example, if I want to create two simulcast streams each with a maximum framerate of 15, and different resolutions (e.g. one stream at half the resolution of the other one), a frameRate: {max: 15} constraint and the scaleResolutionDownBy attribute can suffice for this. 

Similarly, if it is desired to create simulcast streams with the same resolution and different framerates (the scenario that Harald refers to below), this also can be done via track cloning and frameRate constraints, without maxFramerate in RTCRtpEncodingParameters.  At TPAC we discussed the disadvantage of the track cloning approach, which is that the simulcast streams can converge.  However, as Harald notes below, this is still true if we add maxFramerate to RTCRtpEncodingParameters.  Even though the browser in theory now understands that the two streams represent a simulcast, maxFramerate in RTCRtpEncodingParameters produces much the same result as a frameRate constraint.  

So in what simulcast scenario might maxFramerate provide value?  One that comes to mind is a situation in which it is desired to provide a screen share that could be viewed by a PC and a mobile device.  In this situation, gUM constraints could be used to produce a video track with a resolution and maximum frameRate appropriate for the PC, and then resolutionScaleDownBy and maxFrameRate attributes would be set on RTCRtpEncodingParameters to produce an encoding suitable for the mobile device.   

Note that this scenario could also be addressed by track cloning and gUM constraints, but unlike the pure maximum framerate simulcast scenario, there is an advantage to be gained by having the browser know that the two simulcast streams are related - assuming that if the camera resolution changes, it is desired that both streams adjust in resolution, but not exceedthe maximum framerate the device can handle - the PC stream because of the gUM frameRate constraint, and the mobile stream because of the maxFramerate attribute in RTCRtpEncodingParameters. 

So based on the above, I do see a scenario for maxFramerate - but only for a subset of the scenarios that it has been touted for.  

-----Original Message-----
From: Harald Alvestrand [mailto:harald@alvestrand.no] 
Sent: Monday, December 14, 2015 5:48 AM
To: public-webrtc@w3.org
Subject: Consensus call: MaxFramerate for RTCRtpEncodingParameters

At the Sapporo meeting, we had quite a bit of discussion on how to specify framerate restrictions for encodings in a simulcast.

One suggestion was to follow the model of "scaleResolutionDownBy" and specify a divisor that an encoding would use compared to the "main"
encoding. The argument against this was that if the "main" encoding dropped its framerate for any reason, the effect on the other encodings was likely to be different from what the app wanted - if the "main" was
60 fps and the "backup" was 1/4 of that (15 fps), and for some reason (low light?) the "main" dropped to 15 fps, the "backup" would drop to
3.75 fps.

The other suggestion was to specify the max framerate as a "maxFramerate" parameter - which, in the same situation, could lead to multiple encodings with the same framerate (in the same configuration as above, both would send at 15 fps); my impression from the meeting was that there was not a strong consensus one way or the other; some speakers clearly preferred the "maxFramerate" solution, and it wasn't clear that anyone strongly preferred the "scaleFramerateDownBy" solution.

It was, however, clear that people wanted the ability to have encodings with different framerates, so the spec needs to say *something*; based on my perception of the discussion as described above, my impression is that it's more reasonable to specify "maxFramerate" than to specify "scaleFramerateDownBy".

This is a consensus call for adding a maxFramerate parameter to the dictionary "RTCRtpEncodingParameters" in the spec. If you have a preference for or against this addition, please speak up now.

This consensus call runs until Friday, December 18. The chairs will judge consensus based on comments received at that time.

Harald, for the chairs

[Note: The working editors' draft (not the published editors' draft) of the spec already contains this addition. If consensus is against adding it, we'll roll this change back before publishing another editors' draft.]

Received on Tuesday, 15 December 2015 06:16:43 UTC