Re: A Big Proposal: A way to control quality/resolution/framreate/simulcast/layering with RtpSender

On Wed, Feb 19, 2014 at 8:32 PM, Chris Wendt <chris-w3c@chriswendt.net>wrote:

>
> Maybe I’m reading more into this or maybe I’m making it more complex than
> necessary, or maybe i’m stating the obvious, but I think this makes sense.
>
> My originally assumed premise was that you setup a stream with a set of
> parameters and let the implementation map those parameters to video encoder
> rate controls and it runs happily along independently doing adaptive
> bitrate things within the context of the given parameters.
>
> Taking things to the next level, we can imagine having and likely
> requiring application level control, particularly for SVC and simulcast or
> in cases where you have multiple RTPSenders, where we need to control
> across multiple potentially unrelated and independent media streams and
> based on some metrics feedback that, for example, quality is suffering
> given some bitrate constraints, so I should switch to half resolution, or
> no longer send the 3rd enhancement layer or reduce the number of RTPSenders.
> ​​


> So what would be the feedback mechanism here?  Is this up to the
> application developer to monitor metrics themselves if even possible or
> practical?
>
>
​
​Yes, that was part of the original proposal: that we have a feedback
mechanism that allows JS to do more advanced things like that.  I believe
it is possible and practical.  Admittedly, that part of the proposal is
TBD, so we need to work it out a bit more to be sure.
​


> I would propose we think about providing callback hooks based on things
> like min/max bitrate and min/max quality thresholds.  And maybe
> specifically define the basic set.
>
>
​I think polling, like we do for other stats, is probably sufficient.  The
JS should be able to poll for what the current
resolution/quality/framerate/bitrate of each layer is. But if we can think
of some use cases where that's not sufficient, we could think of more than
just polling stats.



> Again, I think the current discussion around the simple set of parameters
> satisfies the static config case, but when we talk about max or min type of
> parameters, I think there needs to be some feedback to notify that
> application when the min/max point is reached so it can act appropriately.


> Thoughts?
>
>
>
> On Feb 19, 2014, at 6:52 PM, Martin Thomson <martin.thomson@gmail.com>
> wrote:
>
> > On 19 February 2014 15:44, Peter Thatcher <pthatcher@google.com> wrote:
> >> I think we still need scale for simulcast and maxBitrate for cases
> where you
> >> want to constrain bandwidth even when it's available.    And priority
> vs.
> >> resources seems pretty similar.
> >
> >
> > I'd like to take simulcast out actually.  I think that aside from some
> > bindings necessary to get playback right, you can achieve simulcast
> > transmission (what we are talking about here) by having multiple
> > tracks with different resolution constraints.  I don't think that
> > means fewer options sadly.
> >
> > Maybe we can also discuss minimums.  I don't think that it's
> > worthwhile having minimum values initially, and maybe not ever, though
> > I'm open to the idea.  And I think that it's a universally applicable
> > thing across all axes.  I can see cases for minimums on all three:
> > frame rate (sign language), resolution (1x1, my image recognition
> > can't deal), and quality (my eyes, ow, my eyes).
>
>