Re: Constraints and Encoding Parameters

I think it's clear we want to avoid the need for track cloning to do
simulcast.  And I think it's clear we want either maxFramerate or
scaleFramerateDownBy.  As Harald points out, if the framerate is not
changing and not degraded, then they are basically the same.  It's only in
the case where the framerate changes (by the source track) or the framerate
is degraded (by the RtpSender) that their behavior differs.

In my opinion, the real question is: when the framerate of the source track
changes or the framerate is degraded, what should happen to different
encodings that have different values of
 maxFramerate/scaleFramerateDownBy?  For example, if we have
[{maxFramerate: 15}, {maxFramerate: 30}] and the camera drops the source
framerate to 12, do we end up with two encodings of 12?   Or one with 6 and
one with 12?

Once we are clear and what we want the behavior to be in that case, I think
it will be clear which of maxFramerate or scaleFramerateDownBy makes more
sense to express that behavior.


By the way, the original consensus call was supposed to continue until Dec,
18th, which has passed.  But then this discussion has continued passed that
date.  So, what happens to the consensus call?











On Mon, Dec 14, 2015 at 10:16 PM, Bernard Aboba <Bernard.Aboba@microsoft.com
> wrote:

> Before expressing an opinion relating to the consensus call, I thought it
> might be useful to have a bit of a discussion about this.
>
> In general, the principle behind RTCRtpEncodingParameters is to avoid
> duplicating functionality in gUM constraints.
>
> Since gUM already supports frameRate constraints (min, max, etc.) before
> adding maxFramerate to RTCRtpEncodingParameters, we should understand what
> scenarios require it.
>
> First off, there are some simulcast scenarios involving maximum framerates
> that do not require maxFramerate in RTCRtpEncodingParameters.  For example,
> if I want to create two simulcast streams each with a maximum framerate of
> 15, and different resolutions (e.g. one stream at half the resolution of
> the other one), a frameRate: {max: 15} constraint and the
> scaleResolutionDownBy attribute can suffice for this.
>
> Similarly, if it is desired to create simulcast streams with the same
> resolution and different framerates (the scenario that Harald refers to
> below), this also can be done via track cloning and frameRate constraints,
> without maxFramerate in RTCRtpEncodingParameters.  At TPAC we discussed the
> disadvantage of the track cloning approach, which is that the simulcast
> streams can converge.  However, as Harald notes below, this is still true
> if we add maxFramerate to RTCRtpEncodingParameters.  Even though the
> browser in theory now understands that the two streams represent a
> simulcast, maxFramerate in RTCRtpEncodingParameters produces much the same
> result as a frameRate constraint.
>
> So in what simulcast scenario might maxFramerate provide value?  One that
> comes to mind is a situation in which it is desired to provide a screen
> share that could be viewed by a PC and a mobile device.  In this situation,
> gUM constraints could be used to produce a video track with a resolution
> and maximum frameRate appropriate for the PC, and then
> resolutionScaleDownBy and maxFrameRate attributes would be set on
> RTCRtpEncodingParameters to produce an encoding suitable for the mobile
> device.
>
> Note that this scenario could also be addressed by track cloning and gUM
> constraints, but unlike the pure maximum framerate simulcast scenario,
> there is an advantage to be gained by having the browser know that the two
> simulcast streams are related - assuming that if the camera resolution
> changes, it is desired that both streams adjust in resolution, but not
> exceedthe maximum framerate the device can handle - the PC stream because
> of the gUM frameRate constraint, and the mobile stream because of the
> maxFramerate attribute in RTCRtpEncodingParameters.
>
> So based on the above, I do see a scenario for maxFramerate - but only for
> a subset of the scenarios that it has been touted for.
>
> -----Original Message-----
> From: Harald Alvestrand [mailto:harald@alvestrand.no]
> Sent: Monday, December 14, 2015 5:48 AM
> To: public-webrtc@w3.org
> Subject: Consensus call: MaxFramerate for RTCRtpEncodingParameters
>
> At the Sapporo meeting, we had quite a bit of discussion on how to specify
> framerate restrictions for encodings in a simulcast.
>
> One suggestion was to follow the model of "scaleResolutionDownBy" and
> specify a divisor that an encoding would use compared to the "main"
> encoding. The argument against this was that if the "main" encoding
> dropped its framerate for any reason, the effect on the other encodings was
> likely to be different from what the app wanted - if the "main" was
> 60 fps and the "backup" was 1/4 of that (15 fps), and for some reason (low
> light?) the "main" dropped to 15 fps, the "backup" would drop to
> 3.75 fps.
>
> The other suggestion was to specify the max framerate as a "maxFramerate"
> parameter - which, in the same situation, could lead to multiple encodings
> with the same framerate (in the same configuration as above, both would
> send at 15 fps); my impression from the meeting was that there was not a
> strong consensus one way or the other; some speakers clearly preferred the
> "maxFramerate" solution, and it wasn't clear that anyone strongly preferred
> the "scaleFramerateDownBy" solution.
>
> It was, however, clear that people wanted the ability to have encodings
> with different framerates, so the spec needs to say *something*; based on
> my perception of the discussion as described above, my impression is that
> it's more reasonable to specify "maxFramerate" than to specify
> "scaleFramerateDownBy".
>
> This is a consensus call for adding a maxFramerate parameter to the
> dictionary "RTCRtpEncodingParameters" in the spec. If you have a preference
> for or against this addition, please speak up now.
>
> This consensus call runs until Friday, December 18. The chairs will judge
> consensus based on comments received at that time.
>
> Harald, for the chairs
>
> [Note: The working editors' draft (not the published editors' draft) of
> the spec already contains this addition. If consensus is against adding it,
> we'll roll this change back before publishing another editors' draft.]
>
>