Re: [webrtc-pc] Constrainable properties on remote tracks are under-specified (#2121)

> I think a lot of this boils down to if you view "applyConstraints()" as means to change configurations of a source or filter/transform what the source provides you with.

It's clearly both. Constraints outsource (no pun intended) *user requirements* from a source to its sink(s); a classic *"put the control surface on the result object"* thing. A `width` might reconfigure the camera, or software downscale, doesn't matter. Tracks are inseparable from their source; multiple heads of it.

It's an API. We're in an ugly spot of asking: *"What behavior should this API produce?"*, when we need to ask: *"What remote track use-cases must be solved (and what's the best API for them)?"*

In the case of https://github.com/w3c/mediacapture-screen-share/issues/31#issuecomment-367200644 we had those use-cases.

> We can either be "defensive", and say "only these constraints are supported for remote tracks" and be explicit about it, or "offensive" and allow all constraints that are still defined in a way that can be applied to remote tracks to be so. ***Example: "width" means the same thing independent of what source produced the frames.***

It doesn't. A *camera track* is inherently and observably different from a *screen-share track* here:
 * [`width`](https://w3c.github.io/mediacapture-screen-share/#dfn-width) on a screen-share track preserves aspect, to not lose information around the edges.
 * [`width`](https://w3c.github.io/mediacapture-main/getusermedia.html#def-width) on a camera track might crop the output, compromising `aspectRatio`, and/or `frameRate`.

There are other differences as well: screenshare tracks [never fire `overconstrained`](https://w3c.github.io/mediacapture-screen-share/#constrainable-properties-for-captured-display-surfaces) and never `mute`. `advanced` and `min` are effectively useless (and provoke `TypeError` in *getDisplayMedia()*). [`resizeMode`](https://w3c.github.io/mediacapture-screen-share/#dfn-resizemode) can be used to detect retina. Many of these behaviors weren't obvious until we specified them.

If "defensive" means specifying how `width` should work on a peer connection track, if at all, and not leaving it to implementor whim, then count me in.

-- 
GitHub Notification of comment by jan-ivar
Please view or discuss this issue at https://github.com/w3c/webrtc-pc/issues/2121#issuecomment-471579204 using your GitHub account

Received on Monday, 11 March 2019 15:09:51 UTC