Re: Constraints and Capabilities API for getUserMedia: more detailed proposal

Le mardi 13 mars 2012 à 12:42 -0400, Dan Burnett a écrit :
> Here is the proposal for Constraints and Capabilities for the getUserMedia draft.

Thanks for sending this out! I'm still on the fence on whether we need
to important all the complexity of the constraints API, but this sounds
like a reasonable starting point (I have some comments on it below.

The getCapabilities() relies on the "trusted" use case, which to me
means a longer-term deployment (since we don't have a good story for
setting up/detecting such a trusted environment), so I'm hoping it would
be developed as a separate module.

> There are definitely some details to work out, particularly around
> which constraints we wish to define in this document.  I also can't
> figure out how to define some of the WebIDL, so I could use some help
> there.
> 
> However, this should better reflect what we agreed to in our last call.


> interface NavigatorUserMedia {
>   void getUserMedia (MediaStreamConstraints? constraints, NavigatorUserMediaSuccessCallback? successCallback, NavigatorUserMediaErrorCallback? errorCallback);
>   MediaStreamCapabilities getCapabilities();
> };

I think getCapabilities() on navigator sounds like too generic; I would
rather see a more specific method name, e.g. getUserMediaCapabilities()
or getMediaCapabilities().

> dictionary MediaStreamConstraints {
>   sequence<MediaStreamConstraint>? mandatory = null;
>   sequence<MediaStreamConstraint>? optional = null;
> };
> 

I'm not clear on how you would ask for a pure-audio or a pure-video
stream with that approach? would there be a generic "audio" and
"video" (presumably boolean) constraint?

> navigator.getUserMedia(
>   {mandatory:[video-min-height:600, video-max-bandwidth:500],
>     optional:[
>       video-max-aspectratio:1.333333333333,
>       video-min-timebetweenrefframes:20,
>       video-min-framerate:30,
>       video-autowhitebalance:on]},
>   gotStream, didntGetStream);

All these video- prefixes are quite repetitive; couldn't they be
factored out in a {video: {max-aspectratio:...,
min-timebetweenframes: ...}}

Maybe min/max ration could be specified likewise (e.g. aspectratio:
{min: ..., max:...}). 

Are there any "native" API in that space we could compare this proposal
to? Does Flash offer that kind of detailed set up?

I think Cullen mentioned that echo cancellation would be dealt through
this contraint API; I don't understand how that would work.
https://www.w3.org/Bugs/Public/show_bug.cgi?id=15747
http://www.w3.org/2011/04/webrtc/track/actions/34

> ===
> Capabilities
> ===
> 
> The return value from getCapabilities() is a JavaScript Object (MediaStreamConstraints) containing, for each device/media stream/channel, a list of MediaStreamConstraint keys and values.  In the trusted scenario, the browser MUST return all relevant constraints of type "min" or "max" as follows:
> - if the constraint is a "min" constraint, it must represent the smallest value of that constraint across all supported streams of the given stream type.  For example, if the constraint is "video-min-width:500", that would mean that there is no video stream with a width smaller than 500 pixels.
> - if the constraint is a "max" constraint, it must represent the largest value of that constraint across all supported streams of the given stream type.  For example, if the constraint is "video-max-width:1000", that would mean that there is no video stream with a width greater than 1000 pixels.
> 
> 
> Example:
> 
> {camera001:{
>   video-min-width:  800,...}
> camera002:{...}
> ...}

Any reason why you chose to use named numbered properties rather than
{cameras: [{video-min-width: 800, ...}, {...}]}

This would help for the WebIDL:
dictionary MediaSTreamCapabilities {
 sequence<VideoConstraints> camera; // or maybe videoInput?
 sequence<AudioConstraints> audio; // or maybe audioInput?
}

where VideoConstraints and AudioConstraints are just objects.

Dom

Received on Tuesday, 13 March 2012 17:42:00 UTC