Constraints 2014

We can change constraints to use WebIDL again while preserving baseline 
functionality.

Basically, we fix the syntax without changing the semantics of 
constraints (much):

Main proposal: require
======================

dictionary Constraints {
     sequence<DOMString> require; // names of required constraints, if any
};

dictionary MediaConstraints : Constraints {
     ConstrainLong width;
     ConstrainLong height;
     ConstrainDouble aspectRatio;
     ConstrainDouble frameRate;
     ...
};

typedef (long or ConstrainLongRange) ConstrainLong;

dictionary ConstrainLongRange {
     long min;
     long max;
};

Clients may list the names of individual constraints they require. For 
example:

   var constraints = {
       require: ["width", "height"],
       width: { min: 640, max: 1280 },
       height: { min: 480, max: 768 },
       aspectRatio: 16/9,
       frameRate: 60,
   };
   navigator.getUserMedia(constraints, success, failure);

GetUserMedia fails if the required constraints are unknown, missing or 
unsatisfied.

Here it fails if none of the sources support a resolution within the 
range (640 to 1280) x (480 to 768) regardless of aspect or frame-rate.

There is no order by default, instead sources are ranked by how many 
constraints they satisfy, with the UA breaking a tie for the default 
choice (which end-users may ultimately override).

This is the core of the proposal.

Two extensions are needed to reach functional parity, but add complexity:


Extension #1: prefer
====================

dictionary Constraints {
     sequence<DOMString> require; // names of required constraints, if any
     sequence<DOMString> prefer;  // names of preferred constraints, if any
};

Clients who prefer an order to their constraints may specify one:

   var constraints = {
       require: ["width", "height"],
       prefer: ["aspectRatio", "frameRate"],
       width: { min: 640, max: 1280 },
       height: { min: 480, max: 768 },
       aspectRatio: 16/9,
       frameRate: 60,
       facingMode: "user",
   };
   navigator.getUserMedia(constraints, success, failure);

Sources are eliminated by required constraints first. The remaining 
sources are ranked by constraints in the preferred order (e.g. 
aspectRatio, then frameRate, then facingMode) using a familiar algorithm [1]

Since prefer is specified, the above example will rank 16:9 60hz 
user-facing sources highest, followed by other 16:9 60hz sources, 
followed by other 16:9 sources, followed by everything else.

GetUserMedia will not fail if preferred constraints are missing, unknown 
or unsatisfied.


Extension #2: ideal
===================

dictionary ConstrainLongRange {
     long min;
     long max;
     long ideal;
};

dictionary ConstrainDoubleRange {
     double min;
     double max;
     double ideal;
};

Ideal expresses a target value within an acceptable range, with values 
closer to the ideal more desirable.

For example, clients may use this to express preference for higher values:

   var constraints =
   {
       aspectRatio: 16/9,
       frameRate: { min: 60, max: 300, ideal: 300 },
   };
   navigator.getUserMedia(constraints, success, failure);

Only min and max need to be satisfied to satisfy a constraint, but 
sources satisfied by the same range are individually ranked by their 
proximity to the ideal. [2]

This example ranks 16/9 sources with frame-rates closest to 300hz 
highest, followed by gradually lower frame-rate 16:9 ones down to 60hz, 
followed by all other aspects 300hz and down.


That's it!
==========

I'd be happy to work with the Editors to incorporate this proposal if 
the TF likes it.


.: Jan-Ivar :.

---

Apendix 1: Fine print (skip!)

[1] Prefer algorithm: When a prefer array is specified, ranking is 
determined by a (familiar) for-each-constraint source-elimination 
algorithm that awards points for each round. This algorithm eliminates 
the "sub-set less than the entire set of remaining sources" that does 
not satisfy a constraint (i.e. if no sources satisfy the constraint, go 
to next constraint, else if one or more sources satisfy the constraint, 
reduce sources to those and go to next constraint). The net result is 
the "preferred set" (the ones with most points).  If, after this, there 
are more constraints to process (e.g. facingMode), then sources in the 
preferred set are individually ranked (earn more points) by how many 
remaining constraints they satisfy. Names already in require or earlier 
in prefer are ignored.

[2] Ideal algorithm: For example, if three sources satisfy the frameRate 
constraint above, the one with the lowest frame-rate among them gets 1/3 
point, the middle one gets 2/3 points and the highest frame-rate one 
gets one point.


Appendix 2: Full WebIDL
=======================

- Everything is defined in standard WebIDL.
- Audio and video constraints co-exist with non-constraints like 
peerIdentity.
- Media-types (audio, video) are inferred from known type-specific 
constraints.
- We re-absorb Constrainable for context needed to define things concretely.
- getNativeSettings() is unneeded. "ideal" resolution etc. is in 
getCapabilities().


dictionary Constraints {
     sequence<DOMString> require; // names of required constraints, if any
     sequence<DOMString> prefer;  // names of preferred constraints, if any
};

dictionary MediaConstraints : Constraints {
     boolean video;
     boolean audio;
     DOMString peerIdentity;
     ConstrainLong width;
     ConstrainLong height;
     ConstrainDouble aspectRatio;
     ConstrainDouble frameRate;
     ConstrainVideoFacingMode facingMode;
     ConstrainDouble volume;
     ConstrainLong sampleRate;
     ConstrainLong sampleSize;
     boolean echoCancelation;
     ConstrainDOMString sourceId;

     // Dictionary may be extended through gUM-specific IANA registry.
};

typedef (long or ConstrainLongRange) ConstrainLong;
typedef (double or ConstrainDoubleRange) ConstraintDouble;
typedef (VideoFacingModeEnum or sequence<VideoFacingModeEnum>) 
ConstrainVideoFacingMode;
typedef (DOMString or sequence<DOMString>) ConstrainDOMString;

dictionary ConstrainLongRange {
     long min;
     long max;
     long ideal;
};

dictionary ConstrainDoubleRange {
     double min;
     double max;
     double ideal;
};

interface MediaStreamTrack : EventTarget {
     MediaConstraints getCapabilities();
     MediaConstraints getConstraints();
     MediaConstraints getSettings();
     void applyConstraints(MediaConstraints constraints,
                           VoidFunction successCallback,
                           ConstraintErrorCallback errorCallback);
              attribute EventHandler onoverconstrained;
     readonly attribute DOMString             kind;
     readonly attribute DOMString             id;
     readonly attribute DOMString             label;
              attribute boolean               enabled;
     readonly attribute boolean               muted;
              attribute EventHandler          onmute;
              attribute EventHandler          onunmute;
     readonly attribute boolean               _readonly;
     readonly attribute boolean               remote;
     readonly attribute MediaStreamTrackState readyState;
              attribute EventHandler          onstarted;
              attribute EventHandler          onended;
     MediaStreamTrack clone();
     void             stop();
};