Re: A Big Proposal: A way to control quality/resolution/framreate/simulcast/layering with RtpSender

My question is: under what circumstances does the app care about high
resolution, low PSNR vs. low resolution, high PSNR?   And the end of the
day, does the user see a difference big enough that the app would care to
tweak this knob?


On Wed, Feb 19, 2014 at 2:45 PM, Chris Wendt <chris-w3c@chriswendt.net>wrote:

> I guess i’m a little confused at your specific definition of resolution
> vs. quality.
>
> resolution, to me means pixel dimensions.  Quality means PSNR type of
> metric.
>
> On Feb 19, 2014, at 5:38 PM, Peter Thatcher <pthatcher@google.com> wrote:
>
> So, something like:
>
> {
>   frameratePriorty: 0.5,
>   resolutionPriority: 0.1,
>   qualityPriority: 1.0
> }
>
> Meaning "I want really high quality, with good framerate, and I don't care
> about resolution."
>
> I actually had just that in an earlier design, but the complexity didn't
> seem worth it.  But it's certainly worth discussing.
>
>
> On Wed, Feb 19, 2014 at 2:16 PM, Martin Thomson <martin.thomson@gmail.com>wrote:
>
>> On 19 February 2014 13:33, Peter Thatcher <pthatcher@google.com> wrote:
>> > So, if I may start a TODO list here, I would include:
>> > 1.  Debate "framerate: 30" vs "bias: framerate"
>> > 2.  Figure out what "quality" means numerically.
>>
>> Sounds like a good place to start.
>>
>> I really only threw the example down as a way to highlight something
>> closer to what Chris suggested, which I think was closer to what I
>> think is reasonable.  I intentionally stole as much as possible from
>> your stuff, because I think that it's on the right track.
>>
>> Maybe there's a simple way to deal with this:
>> 1. Priority determines bandwidth allocation (and might have secondary
>> effects on things like DSCP, but let's pretend that's not going to
>> happen; I'm not going to have to try very hard on that count).
>> 2. Everything else is input to the browser in making the
>> spatial/temporal/quality tradeoff.
>>
>> On the latter, here's a strawman:
>>
>> All "preference" values are between 0 and 1.  0 = don't give me media,
>> 1 = please try to retain what the MediaStreamTrack is providing you
>> exactly, and anything in between is degrees of the same.  Unless we
>> are doing raw video, we will never make it to 1 on the quality axis,
>> but we might make it on the other axes, but otherwise these can be
>> treated fairly similarly at the abstract level.
>>
>> That forms a three dimensional space.  For any given bandwidth, there
>> is a surface in that three-dimensional space with [0, 0, 0] on the
>> opposite side to [1,1,1].  If the surface passes through [1,1,1], or
>> passes the other side of it from the origin, that means it is possible
>> to losslessly transmit the video (and we are probably searching for
>> work).
>>
>> Otherwise, there are three modes based on what is provided by the
>> application:  If an application provides three values, the browser's
>> challenge is to find point on that space that minimizes the distance
>> to the application-provided point.  If the application provides two
>> values, the browser either finds the point of minimum distance to the
>> implied line, or the point of intersection between the surface and the
>> line.  If there is only a single value provided, then it is the point
>> on the intersection between the surface and the implied plane that is
>> closest to [1,1,1].
>>
>> Given a definition of each of these axes, I would expect that browsers
>> would be able to implement something that approximates consistent,
>> subject of course to variations in bandwidth availability testing and
>> other constraints.
>>
>> This also generalizes to more dimensions, but I'd caution against that.
>>
>
>
>