Re: Moving getUserMedia to best-effort

On 07/28/2012 01:18 AM, Travis Leithead wrote:
> I’ve been thinking a lot about Constraints and their current integration
> into getUserMedia, and the points that Rich made in a previous
> discussion:
> http://lists.w3.org/Archives/Public/public-media-capture/2012May/0063.html.
> This also came up again in a recent DAP teleconference.
>
> I like constraints. However, I’m starting to believe that putting them
> front-and-center as parameters to getUserMedia is a bad idea. Rich
> articulates this rather well in his last mail (referenced above), but
> let me put forth my own view below and then offer an actionable suggestion.

I have some problems making my mind up - I would be fine either way. An 
alternative could be to make some very strong wording in the spec about 
risks and issues with using mandatory constraints.

>
> Here’s the goal with constraints (as I see it):
>
> ·Provide a set of optional constraints to help the UA (and ultimately
> end-users) select the best device configuration for the scenario the
> developer would like.
>
> ·Provide a set of mandatory constraints to restrict the ability to get
> media (at all) if the web developer’s scenario cannot be met.
>
> Here’s the potential problem (as I see it):
>
> 1.UAs that run untrusted code (e.g., browsers) won’t provide a
> “getCapabilities” API, so that (at the outset) developers won’t know the
> capabilities that are available for the devices of any given UA. This is
> good and I support that.
>
> 2.Without knowing the capabilities, developers may set a “quality bar”
> (hopefully using optional constraints) to try and get the best results
> for their experience.
>
> 3.Developer may get ornery and use mandatory constraints.
>
> 4.UAs come with all sorts of capabilities and form factors, from mobile
> to desktop to home-security hardware. Mandatory constraints /will
> /cut-off some set of these UAs that don’t have capabilities that meet
> the bar.
>
> 5.These devices/UAs will not like being excluded, and will implement a
> variation of the official spec that makes mandatory constraints optional
> (or some other workaround).
>
> Like I said before: I like constraints. But the above problem leads to
> an exclusionary design, rather than a best-effort design, which has
> always been a hallmark of the web. Also, constraints without
> capabilities is generally problematic.
>
> My proposal is simple (in principle): Move constraints from getUserMedia
> to LocalMediaStream and add a getCapabilities API to LocalMediaStream
> too. Revert getUserMedia to its former best-effort approach (with
> respect to the QoS of either the requested video or audio tracks) and
> use the fact that permission will already have been granted to a
> particular combination of tracks (by the time a LocalMediaStream is
> available) as implicit permission to get the capabilities of the devices
> furnishing the existing tracks.
>
> The previous problem steps now work out as follows:
>
> a.UAs that run untrusted code (e.g., browsers) will provide a
> “getCapabilities” API, but only /after/ granting permission for those
> capabilities to be released along with the media they are providing.
>
> b.Developers can negotiate a level of quality that is applicable and
> relevant to the existing capabilities of the device to try and get the
> best results for their experience.
>
> c.Developer may get ornery and reject the media stream if it doesn’t
> meet some quality bar (but this has always been possible with the stop()
> API) and is less likely under these circumstances.
>
> Since operations that obtain the capabilities and assign the constraints
> will be interacting with the local media resources that are furnishing
> the initial MediaStreamTrack(s) of a LocalMediaStream, and given that
> LocalMediaStream objects are strongly tied to local media resources
> (even though their track lists are mutable over time), I propose the
> following:
>
> interface LocalMediaStream : MediaStream {
>
>      void stop ();
>
>      // proposal ******************************
>
>      MediaStreamConstraints? getUserMediaCapabilities();
>
>      void applyConstraints(MediaStreamConstraints? constraints);
>
> };

I think that something like this should be added. But isn't there a 
fingerprinting issue with exposing all capabilities? An alternative, or 
perhaps complement, that we've been discussing is an API for inspecting 
(introspect?) the MediaStreamTrack. It would not expose capabilities, 
but rather what the track is delivering right now (resolution, 
frame-rate, ....).


As for the applyConstraints: I think the discussion has led me to that 
this is something that should be done on a MediaStreamTrack level, not 
MediaStream.

>
> “getUserMediaCapabilities” will, of course, /only/ report the
> capabilities for all the local media resources that have been granted
> permission previously by getUserMedia.
>
> “applyConstraints” would only work against the local media resources
> that have been granted permission previously by getUserMedia.
> Additionally, this API may have the side-effect of stopping any affected
> existing MediaStreamTrack objects and creating new MediaStreamTrack
> objects (if any) that now conform to the constraints. The new tracks
> could be automatically added to the LocalMediaStream (as if
> videoTracks/audioTracks.add had been called) or furnished in a
> callback/eventhandler.
>
> We could also add events to help the developer understand what happened
> when the constraints were applied (e.g., for error conditions).
>

Received on Saturday, 4 August 2012 13:19:57 UTC