RE: Issue 412: Framerate knob for simulcast

-----Original Message-----
From: Cullen Jennings [mailto:fluffy@iii.ca] 
Sent: Wednesday, December 9, 2015 9:28
To: Martin Thomson <martin.thomson@gmail.com>
Cc: Bernard Aboba <Bernard.Aboba@microsoft.com>; public-webrtc@w3.org
Subject: Re: Issue 412: Framerate knob for simulcast

> On Dec 8, 2015, at 6:00 PM, Martin Thomson <martin.thomson@gmail.com> wrote:
> I don't think that we need to determine the maximum framerate.  This would just be a knob you could tweak to reduce resource usage in the case where you don't *need* more frames.

[MH] I believe there are cases where the application *needs* a simulcast stream to have less frames, e.g. to not overwhelm a constrained receiving client.

> 
> I'm more concerned about how an implementation might meet this limit.
> For instance, if I have a 30 frame per second source and this is set 
> to 25, what would you expect to happen?

[CJ] Some video encoders use the motion compensation in the video compression to interpolate virtual intermediate frames so they can produce 25, some implementations would just drop to 15 fps. Implementations can decide and it may depend on the codec, if they are using hardware, amount of CPU available etc. 

Oh and some sort of broken implementation will produce 25 fps by using an algorithm where each time they need to form an output frame, they just use the most recent input frame - that will make from visual jitter in the motion but more than one crappy product has shipped that. 

[MH] I'm quite happy with leaving this to the implementation to decide, and cross my fingers for them to "do the right thing".

[BA] Since it's a maximum Framerate, any value less than 25 would do.  So the encoder could drop every other frame and provide 15 frames/second.  Since this setting is independent of getUserMedia constraints, it won't have any effect on the source framerate (e.g. don't expect the source to magically adjust to 25 fps).  

[MH] Well technically it could have an effect, but indeed I wouldn't count on it. In the case where all encodings have a max frame rate less than, e.g. half of what the source produces by default, and the source is not connected to any other sink that this PeerConnection, then the implementation could internally feedback this information to the source and get it to produce a framerate more suitable for the consumption. This is also applicable for resolution knobs. I base this on my reading of the non-normative "The model" section of mediacapture spec [1]. 

[1] http://w3c.github.io/mediacapture-main/#the-model-sources-sinks-constraints-and-settings

Received on Wednesday, 9 December 2015 18:30:29 UTC