Re: Constraints and Encoding Parameters

On 28/12/15 21:45, Peter Thatcher wrote:
> I think it's clear we want to avoid the need for track cloning to do
> simulcast.

I agree.

>  And I think it's clear we want either maxFramerate or
> scaleFramerateDownBy.

I think so too. The only reason for me bringing this question up was API 
symmetry.

For a track, it is possible to control resolution (width and height) and 
frame rate.

If the control of simulcast resolution is by scaling the track's 
inherent resolution, it looks odd to me to not adopt that model for 
frame rate as well.


>  As Harald points out, if the framerate is not
> changing and not degraded, then they are basically the same.  It's only
> in the case where the framerate changes (by the source track) or the
> framerate is degraded (by the RtpSender) that their behavior differs.
>
> In my opinion, the real question is: when the framerate of the source
> track changes or the framerate is degraded, what should happen to
> different encodings that have different values of
>   maxFramerate/scaleFramerateDownBy?  For example, if we have
> [{maxFramerate: 15}, {maxFramerate: 30}] and the camera drops the source
> framerate to 12, do we end up with two encodings of 12?   Or one with 6
> and one with 12?
>
> Once we are clear and what we want the behavior to be in that case, I
> think it will be clear which of maxFramerate or scaleFramerateDownBy
> makes more sense to express that behavior.
>
>
> By the way, the original consensus call was supposed to continue until
> Dec, 18th, which has passed.  But then this discussion has continued
> passed that date.  So, what happens to the consensus call?
>
>
>
>
>
>
>
>
>
>
>
> On Mon, Dec 14, 2015 at 10:16 PM, Bernard Aboba
> <Bernard.Aboba@microsoft.com <mailto:Bernard.Aboba@microsoft.com>> wrote:
>
>     Before expressing an opinion relating to the consensus call, I
>     thought it might be useful to have a bit of a discussion about this.
>
>     In general, the principle behind RTCRtpEncodingParameters is to
>     avoid duplicating functionality in gUM constraints.
>
>     Since gUM already supports frameRate constraints (min, max, etc.)
>     before adding maxFramerate to RTCRtpEncodingParameters, we should
>     understand what scenarios require it.
>
>     First off, there are some simulcast scenarios involving maximum
>     framerates that do not require maxFramerate in
>     RTCRtpEncodingParameters.  For example, if I want to create two
>     simulcast streams each with a maximum framerate of 15, and different
>     resolutions (e.g. one stream at half the resolution of the other
>     one), a frameRate: {max: 15} constraint and the
>     scaleResolutionDownBy attribute can suffice for this.
>
>     Similarly, if it is desired to create simulcast streams with the
>     same resolution and different framerates (the scenario that Harald
>     refers to below), this also can be done via track cloning and
>     frameRate constraints, without maxFramerate in
>     RTCRtpEncodingParameters. At TPAC we discussed the disadvantage of
>     the track cloning approach, which is that the simulcast streams can
>     converge.  However, as Harald notes below, this is still true if we
>     add maxFramerate to RTCRtpEncodingParameters.  Even though the
>     browser in theory now understands that the two streams represent a
>     simulcast, maxFramerate in RTCRtpEncodingParameters produces much
>     the same result as a frameRate constraint.
>
>     So in what simulcast scenario might maxFramerate provide value?  One
>     that comes to mind is a situation in which it is desired to provide
>     a screen share that could be viewed by a PC and a mobile device.  In
>     this situation, gUM constraints could be used to produce a video
>     track with a resolution and maximum frameRate appropriate for the
>     PC, and then resolutionScaleDownBy and maxFrameRate attributes would
>     be set on RTCRtpEncodingParameters to produce an encoding suitable
>     for the mobile device.
>
>     Note that this scenario could also be addressed by track cloning and
>     gUM constraints, but unlike the pure maximum framerate simulcast
>     scenario, there is an advantage to be gained by having the browser
>     know that the two simulcast streams are related - assuming that if
>     the camera resolution changes, it is desired that both streams
>     adjust in resolution, but not exceedthe maximum framerate the device
>     can handle - the PC stream because of the gUM frameRate constraint,
>     and the mobile stream because of the maxFramerate attribute in
>     RTCRtpEncodingParameters.
>
>     So based on the above, I do see a scenario for maxFramerate - but
>     only for a subset of the scenarios that it has been touted for.
>
>     -----Original Message-----
>     From: Harald Alvestrand [mailto:harald@alvestrand.no
>     <mailto:harald@alvestrand.no>]
>     Sent: Monday, December 14, 2015 5:48 AM
>     To: public-webrtc@w3.org <mailto:public-webrtc@w3.org>
>     Subject: Consensus call: MaxFramerate for RTCRtpEncodingParameters
>
>     At the Sapporo meeting, we had quite a bit of discussion on how to
>     specify framerate restrictions for encodings in a simulcast.
>
>     One suggestion was to follow the model of "scaleResolutionDownBy"
>     and specify a divisor that an encoding would use compared to the "main"
>     encoding. The argument against this was that if the "main" encoding
>     dropped its framerate for any reason, the effect on the other
>     encodings was likely to be different from what the app wanted - if
>     the "main" was
>     60 fps and the "backup" was 1/4 of that (15 fps), and for some
>     reason (low light?) the "main" dropped to 15 fps, the "backup" would
>     drop to
>     3.75 fps.
>
>     The other suggestion was to specify the max framerate as a
>     "maxFramerate" parameter - which, in the same situation, could lead
>     to multiple encodings with the same framerate (in the same
>     configuration as above, both would send at 15 fps); my impression
>     from the meeting was that there was not a strong consensus one way
>     or the other; some speakers clearly preferred the "maxFramerate"
>     solution, and it wasn't clear that anyone strongly preferred the
>     "scaleFramerateDownBy" solution.
>
>     It was, however, clear that people wanted the ability to have
>     encodings with different framerates, so the spec needs to say
>     *something*; based on my perception of the discussion as described
>     above, my impression is that it's more reasonable to specify
>     "maxFramerate" than to specify "scaleFramerateDownBy".
>
>     This is a consensus call for adding a maxFramerate parameter to the
>     dictionary "RTCRtpEncodingParameters" in the spec. If you have a
>     preference for or against this addition, please speak up now.
>
>     This consensus call runs until Friday, December 18. The chairs will
>     judge consensus based on comments received at that time.
>
>     Harald, for the chairs
>
>     [Note: The working editors' draft (not the published editors' draft)
>     of the spec already contains this addition. If consensus is against
>     adding it, we'll roll this change back before publishing another
>     editors' draft.]
>
>

Received on Tuesday, 29 December 2015 13:19:11 UTC