Re: maxHeight and maxWidth

On Wed, Feb 17, 2016 at 10:26 AM, Randell Jesup <randell-ietf@jesup.org>
wrote:

> Ugh, sent from the wrong address...
>
> On 2/16/2016 9:35 PM, Peter Thatcher wrote:
>
> While thinking about this so more, I thought of another difficulty: if
> resolution is degraded, should that degradation happen before or after the
> max?
>
> In your use case with two layers (1 full, 1 max of 90 pixels tall), what
> happens when the RtpSender degrades the resolution?  Let's say it degrades
> a video frame an original height of 360 pixels down to a height of 180
> pixels.  Clearly, the full layer is now 180 pixels high.  But what should
> it do with the max-height-90-pixel layer be?  Should it become 45 pixels
> tall or stay 90 pixels?
>
>
> 90.
> ​
>
>

>
> If it should stay 90, then what happens if the video further degrades down
> to 1/4 of the original size, such that the full layer is also 90 pixels
> high?  Would they both be 90?
>
>
> yes - but the engine running simulcast should (not spec language!) be
> smart enough to shut off the "full" layer at that point (when the two
> resolutions get "close enough".  Similarly, if you defined a
> maxwidth/height layer of 640x480, and feed in a camera at full res - the
> engine should be smart enough to: for HD input, provide 2 layers, one
> 640x480, one full HD, and if the camera will only provide 640x480 to
> provide only a single layer.  If bandwidth degrades enough, the engine can
> degrade that layer even further.
>
> Similarly, if you provide a ScaleBy of 2, and the bandwidth degrades
> enough, the top layer will (should be) be shut off/starved (though you
> could react by cutting the top-layer resolution, and the lower layer would
> get cur too).  If bandwidth degrades further, you don't (shouldn't IMHO)
> turn on the top layer at a lower res and move the lower layer to 1/2 that,
> you should just degrade the lower layer.
>

> This is part of the logic of the
> bandwidth-allocation-and-resolution-adaption code that lives in the
> browser.  It's not defined by the spec, but should do something
> "reasonable" and not too unexpected given the inputs it's processing.  We
> should *not* try to lock down or even significantly constrain such
> algorithms; just provide them input on what the application would like to
> get out of it.
>

>
> The other way you could go here would be to allow applications to make or
> overrule all these decisions.  This would require defining a complex API
> with a bunch of inputs (which would be hard to standardize across
> browsers), and also to run the algorithm in a worker since you can't have
> it waiting on GC - and even then you might have some control-latency
> issues.  Lets not go there....  And lets not go to "lock down a precise
> algorithm about how the browser/simulcast engine must react" (see jib and
> getUserMedia constraints).
>

​I believe you just said "the browser should do X" but "the browser can do
whatever it wants".  Is that correct?



>
> --
> Randell Jesup, Mozilla
>