Re: maxHeight and maxWidth

On 2/12/2016 6:12 PM, Peter Thatcher wrote:
>
>
> On Fri, Feb 12, 2016 at 2:45 PM, Adam Roach <adam@nostrum.com 
> <mailto:adam@nostrum.com>> wrote:
>
>     On 2/12/16 16:10, Peter Thatcher wrote:
>>     ​ But seeing that code does give me an idea for something even
>>     better; add an event to a video track: onresolutionchanged. 
>>     Then, whenever the video track's resolution changes, the
>>     Javascript can reset the scale:
>

I'll note: onresolutionchanged is actually quite useful regardless of 
this conversation, and answers lots of questions we have to answer today 
with "attach it to a hidden <video> or <canvas> element and poll the 
video size there" (ugh)

>>     ​ It has the downside that a frame or two of the old scale might
>>     escape.
>
>     Yeah, I almost proposed this as an alternative, but identified the
>     race and found it to be unacceptable.
>
>     Doing this pushes the problem down the constituency priority chain
>     from implementors all the way to authors and users. And, make no
>     mistake here: you're proposing user-visible glitches. This is
>     exactly the wrong direction.
>
>
> It's a pause of few frames on a 90-pixel tall video feed when a 
> sceencast is resized or a camera is rotated.  I'm also not a fan of 
> glitches, but that's a very minor glitch. It's doubtful anyone will 
> notice on a 90-pixel tall video (who is going to be constantly looking 
> at a 90-pixel tall screencast, anyway?)

Not so much pause as you get several frames of wrong-resolution video, 
which is actually more jarring than a pause in most cases. Likely this 
is better than what Adam and some of us considered as the implementation 
of this approach, which was the hacky polling method above.  Also 
remember application JS can take "a while" to run, due to the 
single-threaded nature of JS, GC, etc, especially if one has a lot of 
tabs/etc in a single process.  (Sometimes many 1/10's of a second; or 
even worse in some cases/platforms)

Also: it's not just clients that can't deal with non-fixed sizes - it's 
overall UI and application designs based around fixed areas and fixed 
sizes for specific video streams (like thumbnails).  If I have a app 
layout that has a fixed-height bar for remote-user thumbnails (sounds 
pretty familiar...), then having those come in at random sizes is bad.  
Using tricks like above means they'll usually come in at the right 
height, but on changes they'll glitch (which may cause the UI to 
"ripple"/relayout for a fraction, or force the app to constrain the 
layout - which it can do).  So these uses can use the 
onresolutionchanged approach, with visual glitches in a few odd cases.

That said: I think you'll find that most app developers would find max 
width/height intuitive and in many cases more closely matched to what 
they want out of the system (even if it has some edge-case downsides of 
it's own), and without it would mostly not realize they should reset 
scale on resolution changes, followed by questions on discuss-webrtc/etc 
(or just not noticing/ignoring when users trip over it).

> If you want glitch-free scale down and scale up, I believe we already 
> have the controls you need: clone the track, apply constraints, and 
> make 2 RtpSenders.  It's not as convenient as using encoding 
> parameters but it works.

Well, it's not just "convenience", it simply doesn't work as well as the 
two streams fight for bandwidth in a way that a "real" simulcast stream 
doesn't, and in some cases dramatically worse (cases where the total 
bandwidth available is on the order of what the smaller encoding is 
happy with).

-- 
Randell Jesup -- rjesup a t mozilla d o t com
Please please please don't email randell-ietf@jesup.org!  Way too much spam