Re: A Big Proposal: A way to control quality/resolution/framreate/simulcast/layering with RtpSender

So, something like:

{
  frameratePriorty: 0.5,
  resolutionPriority: 0.1,
  qualityPriority: 1.0
}

Meaning "I want really high quality, with good framerate, and I don't care
about resolution."

I actually had just that in an earlier design, but the complexity didn't
seem worth it.  But it's certainly worth discussing.


On Wed, Feb 19, 2014 at 2:16 PM, Martin Thomson <martin.thomson@gmail.com>wrote:

> On 19 February 2014 13:33, Peter Thatcher <pthatcher@google.com> wrote:
> > So, if I may start a TODO list here, I would include:
> > 1.  Debate "framerate: 30" vs "bias: framerate"
> > 2.  Figure out what "quality" means numerically.
>
> Sounds like a good place to start.
>
> I really only threw the example down as a way to highlight something
> closer to what Chris suggested, which I think was closer to what I
> think is reasonable.  I intentionally stole as much as possible from
> your stuff, because I think that it's on the right track.
>
> Maybe there's a simple way to deal with this:
> 1. Priority determines bandwidth allocation (and might have secondary
> effects on things like DSCP, but let's pretend that's not going to
> happen; I'm not going to have to try very hard on that count).
> 2. Everything else is input to the browser in making the
> spatial/temporal/quality tradeoff.
>
> On the latter, here's a strawman:
>
> All "preference" values are between 0 and 1.  0 = don't give me media,
> 1 = please try to retain what the MediaStreamTrack is providing you
> exactly, and anything in between is degrees of the same.  Unless we
> are doing raw video, we will never make it to 1 on the quality axis,
> but we might make it on the other axes, but otherwise these can be
> treated fairly similarly at the abstract level.
>
> That forms a three dimensional space.  For any given bandwidth, there
> is a surface in that three-dimensional space with [0, 0, 0] on the
> opposite side to [1,1,1].  If the surface passes through [1,1,1], or
> passes the other side of it from the origin, that means it is possible
> to losslessly transmit the video (and we are probably searching for
> work).
>
> Otherwise, there are three modes based on what is provided by the
> application:  If an application provides three values, the browser's
> challenge is to find point on that space that minimizes the distance
> to the application-provided point.  If the application provides two
> values, the browser either finds the point of minimum distance to the
> implied line, or the point of intersection between the surface and the
> line.  If there is only a single value provided, then it is the point
> on the intersection between the surface and the implied plane that is
> closest to [1,1,1].
>
> Given a definition of each of these axes, I would expect that browsers
> would be able to implement something that approximates consistent,
> subject of course to variations in bandwidth availability testing and
> other constraints.
>
> This also generalizes to more dimensions, but I'd caution against that.
>