Re: maxHeight and maxWidth

On Sun, Feb 14, 2016 at 9:02 AM, Cullen Jennings (fluffy) <fluffy@cisco.com>
wrote:

>
> I'd like to point out that Poor Man's simulcast does not actually work
> very well for a wide variety of reasons including
>
> 1) lack of synchronization
>

​Can you explain what you mean by synchronization?  In the use case
described in this thread, there is no synchronization between the two video
feeds that I know of.


> 2) runs on separate ports / DTLS sessions which you point out reduce
> success rate
>

​This is clearly solved by bundling the two RtpSenders.​


> 3) poor interaction with browser based congestion controls
>
​
​In what way?  There are per-layer bandwidth controls that should work
equally well on two RtpSenders or one.  Can you explain more what you mean?​


> 4) bugs in browsers
>

Can you be more specific?


>
> Sure I understand that we could fix / work around some of these but
> telling people to use the poor mans solution just does not really meet
> folks needs.
>

If "folks"
​tell me that they must make an RtpSender output 90 pixels tall video and
only 90 pixels tall, then I will tell them to use 2 RtpSenders because:

1.  There is no other way to do it, even with maxHeight, because maxHeight
can't prevent the video from dropping below 90 pixels tall.
2.  I can see no technical disadvantage of doing so.

I'm also generally not convinced by the "won't meet people's needs"
argument.  We'll never hit 100% "folk"s needs in 1.0, and we can't keep
doing this "folks have just one more need" thing going forever.  We have to
start saying "no" to things, especially when the use cases are so
incredibly rare and edge-casey as "I need to simulcast a screencast to a
receiving client that can crop but can't scale video and needs exactly 90
pixels and my SFU also can't scale and I won't use 2 RtpSenders because I
can't use BUNDLE and there might be browser bugs".




>
>
> > On Feb 12, 2016, at 1:36 PM, Peter Thatcher <pthatcher@google.com>
> wrote:
> >
> > So, the use case really is "I want to simulcast to a forwarding unit
> that forwards to other units that can't handle more than 90 pixels tall and
> can't resize."
> >
> > If you don't have to simulcast, then this isn't needed because you can
> clone the track and apply new constrains on the height.  And even if you do
> want to simulcast, you could still do that as the "poor man's simulcast".
> >
> > As for a receiver that has to receive exactly 90 pixels because it can't
> resize: frankly, I don't have much sympathy for such an endpoint.  It's not
> going to find a 90-pixel-tall screencast very useful anyway (you'll be able
> to see almost nothing).  And if it can't handle scaling or rotation, it's
> not going to be very happy with the video that comes from a phone that
> rotated it's camera since it will mostly be seeing black bars anyway.
> >
> > This sounds like such an edge case to me that I'd say that "poor man's
> simulcast" is good enough:  clone the track and create two RtpSenders.
> >
> >
> > We've discussed the downsides of maxHeight/maxWidth in many f2f
> meetings, so we can just go dig up the notes from before.  But I recall the
> case against maxHeight and maxWidth is the complexity and ambiguity that it
> introduces, as well as the duplication of control.  You can already control
> the height and width of the track via constraints on the track.  And if we
> did add a maxHeight and the frame is too big, it's not clear what should
> happen: scaling, cropping, pillar boxes, etc.
> >
> > I realize "poor man's simulcast" isn't ideal, and we added RIDs and
> encoding parameters and all of that to avoid it, but if we can cover 99% of
> use cases with this, I don't think it's worth it to add the complexity of
> maxWidth/maxHeight just for some weird edge case that can be handled just
> fine with "poor man's simulcast".
> >
> >
> > On Wed, Feb 10, 2016 at 11:28 AM, Adam Roach <adam@nostrum.com> wrote:
> > On 10/12/15 10:37, Peter Thatcher wrote:
> >> ​ My point was that if you think we should have ".maxWidth" and
> ".maxHeight" in RTCRtpEncodingParameters, that's a separate proposal.   So,
> if you want it, please propose it.
> >
> > This didn't seem particularly important at the time, but we've gotten
> some implementor feedback that indicates otherwise.
> >
> > Specifically, we've heard from implementors that the use of the current
> proposed (sole) control for resizing simulcast streams is problematic in
> the face of streams that can change size (such as captured windows, or
> rotating cameras on mobile devices).
> >
> > The scenario under consideration is one in which each media source
> generates two streams: one full-size, and one which must be exactly 90
> pixels tall. These are being sent to a media switch, which has no ability
> to decode and re-encode streams. Some of the recipients (non-web-browsers)
> do not have the ability to resize incoming streams.
> >
> > In the optimal (and admittedly typical) case, would be possible to do
> something like this:
> >
> >   params.encodings[1].scaleResolutionDownBy =
> videoTrack.getSettings().height / 90;
> >
> > Where this falls apart is when the height of the video track *changes*.
> >
> > By contrast, if they could simply do something like:
> >
> >   params.encodings[1].maxHeight = 90;
> >
> > Then the problem goes away.
> >
> > This seems relatively straightforward to spec out, and -- given that we
> see real implementor need for it -- I think it makes sense to go ahead and
> add it at this point.
> >
> > So, what are the arguments against maxHeight and maxWidth?
> >
> > /a
> >
>
>