Constraints and Capabilities API for getUserMedia: more detailed proposal

Group,

Here is the proposal for Constraints and Capabilities for the getUserMedia draft.
There are definitely some details to work out, particularly around which constraints we wish to define in this document.  I also can't figure out how to define some of the WebIDL, so I could use some help there.

However, this should better reflect what we agreed to in our last call.

-- dan



interface NavigatorUserMedia {
  void getUserMedia (MediaStreamConstraints? constraints, NavigatorUserMediaSuccessCallback? successCallback, NavigatorUserMediaErrorCallback? errorCallback);
  MediaStreamCapabilities getCapabilities();
};

dictionary MediaStreamConstraints {
  sequence<MediaStreamConstraint>? mandatory = null;
  sequence<MediaStreamConstraint>? optional = null;
};

// I don't know how to write the WebIDL definition for MediaStreamCapabilities -- see the getCapabilities description farther down



A MediaStreamConstraint is a key-value pair where the key MUST be a valid registered constraint name in the IANA-hosted RTCWeb Media Constraints registry [1] and the value SHOULD be as defined in the associated reference[s] given in the registry.

===
Constraints
===

When given constraints in the getUserMedia() call, the browser MUST conform to the semantics of the following algorithm when selecting the media stream(s):

1. Define the following:
  a. the set of media types (at the time of writing, only "audio" and "video" are available) addressed by the constraints to be the RequestedMediaTypes.
  b. the FinalSet to be (initially) empty.
  c. the FirstPassSet to be all possible streams the browser can return.
2. For each constraint key-value pair in the mandatory sequence, in order,
  a. Remove from the FirstPassSet any streams of the constraint's media type that cannot satisfy the value given for the constraint.
  b. If the FirstPassSet no longer contains at least one stream for every RequestedMediaType, call the errorCallback with an error indicating failure to satisfy the current mandatory constraint and naming the constraint, then exit.  Otherwise, continue with the next mandatory constraint.
3. For each media type in the RequestedMediaTypes,
  a. Define the CandidateSet to be the subset of the FirstPassSet that match the current media type.
  b. Define the SecondPassSet to be the current contents of the CandidateSet.
  c. For each constraint key-value pair in the optional sequence that are for the current media type, in order,
    1. Remove from the SecondPassSet any streams that cannot satisfy the value given for the constraint.
    2. If the SecondPassSet is now empty, define the SecondPassSet to be the current contents of the Candidate Set.  Otherwise, define the CandidateSet to be the current contents of the SecondPassSet.
  d. Select one stream from the CandidateSet to add to the FinalSet. 
4. Call the successCallback with the list of streams in the FinalSet.

In step 3d the decision of which stream to choose from the CandidateSet is completely up to the browser.  Unless and until a new set of constraints is provided, the browser MAY change its choice of stream at any point, provided that it notifies the application code by again calling the successCallback with the resulting new FinalSet.  It may wish to do this, for example, if the user interface or network congestion changes.

Example:

navigator.getUserMedia(
  {mandatory:[video-min-height:600, video-max-bandwidth:500],
    optional:[
      video-max-aspectratio:1.333333333333,
      video-min-timebetweenrefframes:20,
      video-min-framerate:30,
      video-autowhitebalance:on]},
  gotStream, didntGetStream);


===
Capabilities
===

The return value from getCapabilities() is a JavaScript Object (MediaStreamConstraints) containing, for each device/media stream/channel, a list of MediaStreamConstraint keys and values.  In the trusted scenario, the browser MUST return all relevant constraints of type "min" or "max" as follows:
- if the constraint is a "min" constraint, it must represent the smallest value of that constraint across all supported streams of the given stream type.  For example, if the constraint is "video-min-width:500", that would mean that there is no video stream with a width smaller than 500 pixels.
- if the constraint is a "max" constraint, it must represent the largest value of that constraint across all supported streams of the given stream type.  For example, if the constraint is "video-max-width:1000", that would mean that there is no video stream with a width greater than 1000 pixels.


Example:

{camera001:{
  video-min-width:  800,
  video-max-width:  1024,
  video-min-height:  600,
  video-max-height:  768,
  video-min-aspectratio:  1.333333333333,
  video-max-aspectratio:  1.333333333333,
  video-min-framerate:  24,
  video-max-framerate:  60,
  video-min-pixelrate:  15,
  video-max-pixelrate:  47,
  video-min-timebetweenkeyframes;  20,
  video-max-timebetweenkeyframes:  40,
  video-min-bandwidth:  1.5,
  video-max-bandwidth:  3.5},
camera002:{
  video-min-width:  1600,
  video-max-width:  1920,
  video-min-height:  1080,
  video-max-height:  1200,
  video-min-aspectratio:  1.33333333333,
  video-max-aspectratio:  1.77777777777,
  video-min-framerate:  24,
  video-max-framerate:  120,
  video-min-pixelrate:  57.6,
  video-max-pixelrate:  248,
  video-min-timebetweenkeyframes;  20,
  video-max-timebetweenkeyframes:  40,
  video-min-bandwidth:  8,
  video-max-bandwidth:  29.4},
audio001:{
  audio-min-bandwidth:  1.4,
  audio-max-bandwidth:  128,
  audio-min-mos:  2,
  audio-max-mos:  5,
  audio-min-codinglatency:  10,
  audio-max-codinglatency:  50,
  audio-min-samplingrate:  8000,
  audio-max-samplingrate:  48000}}


[1] http://tools.ietf.org/html/draft-burnett-rtcweb-constraints-registry-00

Received on Tuesday, 13 March 2012 16:43:00 UTC