Re: maxHeight and maxWidth

On Sat, Feb 13, 2016 at 12:04 AM, Randell Jesup <randell-ietf@jesup.org>
wrote:

> On 2/12/2016 6:12 PM, Peter Thatcher wrote:
>
>
>
> On Fri, Feb 12, 2016 at 2:45 PM, Adam Roach <adam@nostrum.com> wrote:
>
>> On 2/12/16 16:10, Peter Thatcher wrote:
>>
>> ​ But seeing that code does give me an idea for something even better;
>> add an event to a video track: onresolutionchanged.  Then, whenever the
>> video track's resolution changes, the Javascript can reset the scale:
>>
>>
> I'll note: onresolutionchanged is actually quite useful regardless of this
> conversation, and answers lots of questions we have to answer today with
> "attach it to a hidden <video> or <canvas> element and poll the video size
> there" (ugh)
>
> ​ It has the downside that a frame or two of the old scale might escape.
>>
>>
>> Yeah, I almost proposed this as an alternative, but identified the race
>> and found it to be unacceptable.
>>
>> Doing this pushes the problem down the constituency priority chain from
>> implementors all the way to authors and users. And, make no mistake here:
>> you're proposing user-visible glitches. This is exactly the wrong direction.
>>
>
> It's a pause of few frames on a 90-pixel tall video feed when a sceencast
> is resized or a camera is rotated.  I'm also not a fan of glitches, but
> that's a very minor glitch.  It's doubtful anyone will notice on a 90-pixel
> tall video (who is going to be constantly looking at a 90-pixel tall
> screencast, anyway?)
>
>
> Not so much pause as you get several frames of wrong-resolution video,
>

​If the receiving endpoint can scale, then it will do so before showing the
user..  It it can't, then the frame won't be seen.  Either way, the user
won't see a wrong resolution.


> which is actually more jarring than a pause in most cases.  Likely this is
> better than what Adam and some of us considered as the implementation of
> this approach, which was the hacky polling method above.  Also remember
> application JS can take "a while" to run, due to the single-threaded nature
> of JS, GC, etc, especially if one has a lot of tabs/etc in a single
> process.  (Sometimes many 1/10's of a second; or even worse in some
> cases/platforms)
>
> Also: it's not just clients that can't deal with non-fixed sizes - it's
> overall UI and application designs based around fixed areas and fixed sizes
> for specific video streams (like thumbnails).  If I have a app layout that
> has a fixed-height bar for remote-user thumbnails (sounds pretty
> familiar...), then having those come in at random sizes is bad.  Using
> tricks like above means they'll usually come in at the right height, but on
> changes they'll glitch (which may cause the UI to "ripple"/relayout for a
> fraction, or force the app to constrain the layout - which it can do).  So
> these uses can use the onresolutionchanged approach, with visual glitches
> in a few odd cases.
>
> That said: I think you'll find that most app developers would find max
> width/height intuitive and in many cases more closely matched to what they
> want out of the system (even if it has some edge-case downsides of it's
> own),
>

​I hope we can all remember that we are only considering
scaleResolutionDownToHeight/Width for the encoding parameters *for
simulcast scenarios*.  If simulcast isn't part of the scenario, then
there's no need for these (or maxFramerate, or scaleResolutionDownBy,
etc).  And most app developers never deal with simulcast.   Anything we add
for simulcast is already well in the realm of an advanced use case.
​

> and without it would mostly not realize they should reset scale on
> resolution changes, followed by questions on discuss-webrtc/etc (or just
> not noticing/ignoring when users trip over it).
>
> If you want glitch-free scale down and scale up, I believe we already have
> the controls you need: clone the track, apply constraints, and make 2
> RtpSenders.  It's not as convenient as using encoding parameters but it
> works.
>
>
> Well, it's not just "convenience", it simply doesn't work as well as the
> two streams fight for bandwidth in a way that a "real" simulcast stream
> doesn't, and in some cases dramatically worse (cases where the total
> bandwidth available is on the order of what the smaller encoding is happy
> with).
>

​I don't see how this has anything to do with bandwidth.  Two layers on one
RtpSender or two RtpSenders with one layer each shouldn't matter much at
all to the bandwidth allocator.  Certainly a bandwidth allocator should be
smart enough to handle two RtpSenders (two cameras) with two layers each,
so why not two with one layer each?  I don't see what the problem here is.

But as I've mentioned before, scaleResolutionDownToHeight/Width does not
solve the originally proposed use case in this thread.  It cannot satisfy
the requirement to always send 90 pixels tall.  Even with
scaleResolutionDownToHeight/Width, you'd still need to have two RtpSenders
to deal with the case where the screencast is less than 90 pixels tall.
​


>
>
> --
> Randell Jesup -- rjesup a t mozilla d o t com
> Please please please don't email randell-ietf@jesup.org!  Way too much spam
>
>