Re: maxHeight and maxWidth

So, the use case really is "I want to simulcast to a forwarding unit that
forwards to other units that can't handle more than 90 pixels tall and
can't resize."

If you don't have to simulcast, then this isn't needed because you can
clone the track and apply new constrains on the height.  And even if you do
want to simulcast, you could still do that as the "poor man's simulcast".

As for a receiver that has to receive exactly 90 pixels because it can't
resize: frankly, I don't have much sympathy for such an endpoint.  It's not
going to find a 90-pixel-tall screencast very useful anyway (you'll be able
to see almost nothing).  And if it can't handle scaling or rotation, it's
not going to be very happy with the video that comes from a phone that
rotated it's camera since it will mostly be seeing black bars anyway.

This sounds like such an edge case to me that I'd say that "poor man's
simulcast" is good enough:  clone the track and create two RtpSenders.


We've discussed the downsides of maxHeight/maxWidth in many f2f meetings,
so we can just go dig up the notes from before.  But I recall the case
against maxHeight and maxWidth is the complexity and ambiguity that it
introduces, as well as the duplication of control.  You can already control
the height and width of the track via constraints on the track.  And if we
did add a maxHeight and the frame is too big, it's not clear what should
happen: scaling, cropping, pillar boxes, etc.

I realize "poor man's simulcast" isn't ideal, and we added RIDs and
encoding parameters and all of that to avoid it, but if we can cover 99% of
use cases with this, I don't think it's worth it to add the complexity of
maxWidth/maxHeight just for some weird edge case that can be handled just
fine with "poor man's simulcast".


On Wed, Feb 10, 2016 at 11:28 AM, Adam Roach <adam@nostrum.com> wrote:

> On 10/12/15 10:37, Peter Thatcher wrote:
>
> ​ My point was that if you think we should have ".maxWidth" and
> ".maxHeight" in RTCRtpEncodingParameters, that's a separate proposal.   So,
> if you want it, please propose it.
>
>
> This didn't seem particularly important at the time, but we've gotten some
> implementor feedback that indicates otherwise.
>
> Specifically, we've heard from implementors that the use of the current
> proposed (sole) control for resizing simulcast streams is problematic in
> the face of streams that can change size (such as captured windows, or
> rotating cameras on mobile devices).
>
> The scenario under consideration is one in which each media source
> generates two streams: one full-size, and one which must be exactly 90
> pixels tall. These are being sent to a media switch, which has no ability
> to decode and re-encode streams. Some of the recipients (non-web-browsers)
> do not have the ability to resize incoming streams.
>
> In the optimal (and admittedly typical) case, would be possible to do
> something like this:
>
>   params.encodings[1].scaleResolutionDownBy =
> videoTrack.getSettings().height / 90;
>
> Where this falls apart is when the height of the video track *changes*.
>
> By contrast, if they could simply do something like:
>
>   params.encodings[1].maxHeight = 90;
>
> Then the problem goes away.
>
> This seems relatively straightforward to spec out, and -- given that we
> see real implementor need for it -- I think it makes sense to go ahead and
> add it at this point.
>
> So, what are the arguments against maxHeight and maxWidth?
>
> /a
>

Received on Friday, 12 February 2016 20:37:44 UTC