Re: [webrtc-pc] Define which constraints are applicable in WebRTC (#2188)

(With chair hat off) I'm not personally ready to concede (yet) that all types of tracks should have constrainable properties—`element.captureStream()`?

Constraints are an interoperability cop-out: a license for browsers to act differently, as long as they state their [capabilities](https://w3c.github.io/mediacapture-main/getusermedia.html#dom-constrainablepattern-getcapabilities) (which are allowed by specs to vary astronomically). Even its original purview of camera device discovery seems largely moot thanks to modern down-scaling.

That said, it seems hard to argue against the four points you make being useful, except perhaps that these properties are [already observable using clever JS](https://jsfiddle.net/jib1/7df5e0ts/) through e.g. `element.videoWidth` and `videoHeight`.

It seems too late to argue for as simpler surface—`track.width` and `track.height` anyone?

So maybe we're stuck with the vestigial surface, but we use only part of it, like screen capture did. 

> If a web application sets a width constraint of 640 for a remote track, I would hope all browsers to exhibit the same behaviour if the network decoded frames go from 1000 to 100 and then back to 1000 a minute after.
> 
> If we forbid upsampling, I guess calling getCapabilities would for instance give the current allowed downsampling range which would go from [1, 1000] to [1, 100] and then back to [1, 1000].
> 
> The same principle would seem to apply for window screen share tracks.

A desirable goal, but would require stronger normative language than constraints imply. 

Screen-capture probably has the [strongest language](https://w3c.github.io/mediacapture-screen-share/#downscaling-and-frame-decimation) so far on this, and even it uses SHOULD:

 *"For the purposes of the SelectSettings algorithm, the user agent SHOULD consider all possible combinations of downscaled dimensions that preserve the aspect ratio of the original display surface (to the nearest pixel), and frame rates available through frame decimation, as available settings dictionaries."*

It also says: *"The user agent MUST NOT upscale the captured output"* which I think is good.

But then it says: *"The user agent MUST NOT crop the captured output."*, which makes perfect sense for screen sharing, but differs from camera. So sources behave differently. A remote track could be either, so I'd say MUST NOT crop.

-- 
GitHub Notification of comment by jan-ivar
Please view or discuss this issue at https://github.com/w3c/webrtc-pc/pull/2188#issuecomment-515535854 using your GitHub account

Received on Friday, 26 July 2019 17:28:03 UTC