Re: Constraints and Encoding Parameters

Thanks Bernard for describing a use case.

Though, looking at http://rawgit.com/w3c/webrtc-pc/master/webrtc.html, 
it seems a bit strange to specify frameRate as a value, while resolution 
is a scaling factor of the track's "native" (as controlled by 
constraints) resolution.

We talked a bit about this at TPAC, but it was not entirely clear to me 
why frame rate should be treated differently than resolution. Sure, you 
can argue that if no frameRate constraint is applied you don't know what 
to scale from - but the same is true for resolution.

Stefan

On 15/12/15 07:18, Bernard Aboba wrote:
> Before expressing an opinion relating to the consensus call, I
> thought it might be useful to have a bit of a discussion about this.
>
> In general, the principle behind RTCRtpEncodingParameters is to avoid
> duplicating functionality in gUM constraints.
>
> Since gUM already supports frameRate constraints (min, max, etc.)
> before adding maxFramerate to RTCRtpEncodingParameters, we should
> understand what scenarios require it.
>
> First off, there are some simulcast scenarios involving maximum
> framerates that do not require maxFramerate in
> RTCRtpEncodingParameters.  For example, if I want to create two
> simulcast streams each with a maximum framerate of 15, and different
> resolutions (e.g. one stream at half the resolution of the other
> one), a frameRate: {max: 15} constraint and the scaleResolutionDownBy
> attribute can suffice for this.
>
> Similarly, if it is desired to create simulcast streams with the same
> resolution and different framerates (the scenario that Harald refers
> to below), this also can be done via track cloning and frameRate
> constraints, without maxFramerate in RTCRtpEncodingParameters.  At
> TPAC we discussed the disadvantage of the track cloning approach,
> which is that the simulcast streams can converge.  However, as Harald
> notes below, this is still true if we add maxFramerate to
> RTCRtpEncodingParameters.  Even though the browser in theory now
> understands that the two streams represent a simulcast, maxFramerate
> in RTCRtpEncodingParameters produces much the same result as a
> frameRate constraint.
>
> So in what simulcast scenario might maxFramerate provide value?  One
> that comes to mind is a situation in which it is desired to provide a
> screen share that could be viewed by a PC and a mobile device.  In
> this situation, gUM constraints could be used to produce a video
> track with a resolution and maximum frameRate appropriate for the PC,
> and then resolutionScaleDownBy and maxFrameRate attributes would be
> set on RTCRtpEncodingParameters to produce an encoding suitable for
> the mobile device.
>
> Note that this scenario could also be addressed by track cloning and
> gUM constraints, but unlike the pure maximum framerate simulcast
> scenario, there is an advantage to be gained by having the browser
> know that the two simulcast streams are related - assuming that if
> the camera resolution changes, it is desired that both streams adjust
> in resolution, but not exceedthe maximum framerate the device can
> handle - the PC stream because of the gUM frameRate constraint, and
> the mobile stream because of the maxFramerate attribute in
> RTCRtpEncodingParameters.
>
> So based on the above, I do see a scenario for maxFramerate - but
> only for a subset of the scenarios that it has been touted for.
>
> -----Original Message----- From: Harald Alvestrand
> [mailto:harald@alvestrand.no] Sent: Monday, December 14, 2015 5:48
> AM To: public-webrtc@w3.org Subject: Consensus call: MaxFramerate for
> RTCRtpEncodingParameters
>
> At the Sapporo meeting, we had quite a bit of discussion on how to
> specify framerate restrictions for encodings in a simulcast.
>
> One suggestion was to follow the model of "scaleResolutionDownBy" and
> specify a divisor that an encoding would use compared to the "main"
> encoding. The argument against this was that if the "main" encoding
> dropped its framerate for any reason, the effect on the other
> encodings was likely to be different from what the app wanted - if
> the "main" was 60 fps and the "backup" was 1/4 of that (15 fps), and
> for some reason (low light?) the "main" dropped to 15 fps, the
> "backup" would drop to 3.75 fps.
>
> The other suggestion was to specify the max framerate as a
> "maxFramerate" parameter - which, in the same situation, could lead
> to multiple encodings with the same framerate (in the same
> configuration as above, both would send at 15 fps); my impression
> from the meeting was that there was not a strong consensus one way or
> the other; some speakers clearly preferred the "maxFramerate"
> solution, and it wasn't clear that anyone strongly preferred the
> "scaleFramerateDownBy" solution.
>
> It was, however, clear that people wanted the ability to have
> encodings with different framerates, so the spec needs to say
> *something*; based on my perception of the discussion as described
> above, my impression is that it's more reasonable to specify
> "maxFramerate" than to specify "scaleFramerateDownBy".
>
> This is a consensus call for adding a maxFramerate parameter to the
> dictionary "RTCRtpEncodingParameters" in the spec. If you have a
> preference for or against this addition, please speak up now.
>
> This consensus call runs until Friday, December 18. The chairs will
> judge consensus based on comments received at that time.
>
> Harald, for the chairs
>
> [Note: The working editors' draft (not the published editors' draft)
> of the spec already contains this addition. If consensus is against
> adding it, we'll roll this change back before publishing another
> editors' draft.]
>
>

Received on Tuesday, 22 December 2015 13:12:54 UTC