RE: Relation between a track's constraints/settings and how it is transmitted

Stefan said: 

"I was actually at the step before. You are right in that constraints are applied on the track before it is input to the encoder. But I see no text saying that what the encoder does (and what is the characteristics of the track at the remote endpoint) does have to comply to that in any way.

[BA] Section 5.2.4 defines the attributes within the RTCRtpEncodingParameters dictionary.  So there is some text describing what the encoder does - but perhaps not enough. 

Just as an example: let's say I constrain a local track to height 1080 and framerate 60. There is (as I read the spec) no text saying the encoder should encode something that can be decoded to 1080@60; the UA could at its discretion - even if it has a good network connection and a lot of resources - encode to 360@15. And we have no "overconstrained" 
event to tell the app (even though I guess it can find out from the stats).

[BA] RTCRtpEncodingParameters.scaleResolutionDownBy has a default value of 1.0 (see Section 5.2.3), so if scaleResolutionDownBy is not set to an alternative value, the instruction to the encoder is to attempt to encode the resolution supplied by the local track. 

If RTCRtpEncodingParameters.maxFramerate is not set, the encoder should attempt to encode the framerate supplied by the local track (since there is no maximum framerate indicating otherwise).  

In the event of congestion, it may not be possible for the encoder to carry out the RTCRtpEncodingParameters instructions.  For example, if scaleResolutionDownBy is set
to 1.0, but parameters.degradationPreference is set to "maintain-framerate", then resolution will degrade first.  getParameters() will return parameters.encodings[j].scaleResolutionDownBy set to 1.0.  Within the Statistics API, RTCMediaStreamTrackStats.frameHeight and frameWidth provide information on the
resolution that is actually being sent.    Currently, is no "overconstrained" event for the encoder. 

Similarly, if parameters.degradationPreference is set to "maintain-resolution", in the event of congestion, framerate will degrade first.  getParameters() will return the value of
Parameters.encodings[j].maxFramerate that was originally set.   Within the Statistics API, RTCMediaStreamTrackStats.framesPerSecond provides the frames per second that are sent. 

Of course, congestion could get so bad that both framerate and resolution degrades, regardless of the value of parameters.degradationPreference. 

Stefan also said: 
"So my question was basically if we should have text saying the UA should (try to) honor the track constraints when setting up encoders."

[BA] Feel free to suggest some text.

Received on Tuesday, 19 January 2016 20:10:14 UTC