RE: Moving getUserMedia to best-effort

Don't optional constraints handle the case where the developer would like to have a certain feature, but is willing to make do if it's not available?

- Jim

-----Original Message-----
From: Travis Leithead [mailto:travis.leithead@microsoft.com] 
Sent: Monday, July 30, 2012 2:05 PM
To: Harald Alvestrand; public-media-capture@w3.org
Subject: RE: Moving getUserMedia to best-effort

> From: Harald Alvestrand [mailto:harald@alvestrand.no]
>> On 07/27/2012 04:18 PM, Travis Leithead wrote:
>> Like I said before: I like constraints. But the above problem leads 
>> to an exclusionary design, rather than a best-effort design, which 
>> has always been a hallmark of the web. Also, constraints without 
>> capabilities is generally problematic.
>
> I don't see that logic as compelling.
> A competing logic is:
>
> - Deveopers use mandatory constraints exactly in order to exclude some 
> devices, because they know that users on those devices will have a bad 
> experience
> - Some UA implementors will design non-conformant UAs in order to run 
> on those devices
> - Some users will have a bad experience
>
> Why should we make it hard to decide who to blame in this situation?

It's not so much a question of who to blame, but rather being able to give developers a chance to see the capabilities of an approved device(s) _before_ blindly excluding the user from the experience. The benefit over the existing model is that there may be an opportunity for negotiation once the capabilities are known rather than a flat denial of service.

The only difference in the end-user experience is that the user will be prompted to approve a microphone/camera for use with the site _before_ being told that their hardware doesn't meet the bar.

 
>> My proposal is simple (in principle): Move constraints from 
>> getUserMedia to LocalMediaStream
>
> This would mean that you can't apply constraints up front, you can only apply constraints after getting access to the device. Why is that an advantage?
>
>> [...]
>> Since operations that obtain the capabilities and assign the 
>> constraints will be interacting with the local media resources that 
>> are furnishing the initial MediaStreamTrack(s) of a LocalMediaStream, 
>> and given that LocalMediaStream objects are strongly tied to local 
>> media resources (even though their track lists are mutable over 
>> time),  I propose the following:
>> 
>> interface LocalMediaStream : MediaStream {
>>    void stop ();
>>    // proposal ******************************
>>    MediaStreamConstraints? getUserMediaCapabilities();
>>    void applyConstraints(MediaStreamConstraints? constraints);  };
>> 
>> "getUserMediaCapabilities" will, of course, only report the 
>> capabilities for all the local media resources that have been granted 
>> permission previously by getUserMedia.
>> 
>> "applyConstraints" would only work against the local media resources 
>> that have been granted permission previously by getUserMedia.
>> Additionally, this API may have the side-effect of stopping any 
>> affected existing MediaStreamTrack objects and creating new 
>> MediaStreamTrack objects (if any) that now conform to the 
>> constraints. The new tracks could be automatically added to the 
>> LocalMediaStream (as if videoTracks/audioTracks.add had been called) 
>> or furnished in a callback/eventhandler.
>>
>> We could also add events to help the developer understand what 
>> happened when the constraints were applied (e.g., for error conditions).
>
> I actually see this API as making a lot of sense for *changing* media capabilities on a stream (which may succeed or may fail, so you need a return value or callback from applyConstraint).
>
> But the only difference I see between
> getUserMedia(function(flow) {
>  if (flow.applyConstraints(constraints)) {
>     handleSuccess()
>  } else
>     handleFailure()
>   }
> }
>
> and
>
> flow = getUserMedia(constraints, handleSuccess, handleFailure)
>
> is that you enforce that the user has to be prompted for permission to the device before rejecting it.
>
> I don't see the advantage in that.

To be certain, getUserMedia's options were originally intended to be mandatory constraints, e.g., { video: true}, is a mandatory constraint on the existence of a video capture device in the UA--when it cannot be satisfied, the NavigatorUserMediaErrorCallback (NUMEC?) is invoked without bothering the user.

I'm happy with this basic level of constraint being present as there is no workaround for the application in these scenarios. However, if "video dimensions" or "audio bit-rate" become a mandatory constraint, then it just seems.... over constrained.

Perhaps only the most primitive of mandatory constraints will be defined in this spec, but my worry is that every defined optional constraint will also be available as a mandatory constraint (which makes sense, but then leads to scenarios that "depend upon" very exclusive mandatory constraints). I'm not sure we can avoid this, which is why I'd rather push the application of constraints till the user's media capabilities can be safely discovered.

On the other hand, my proposal encourages the discovery of actual capabilities before applying constraints (which you skipped in your code example above). It also supports changing the configuration of the user media, which is Local Media Requirement 8 (http://dvcs.w3.org/hg/dap/raw-file/071234829689/media-stream-capture/scenarios.html#requirements).

Received on Monday, 30 July 2012 18:19:55 UTC