Re: WebIDL-compatible syntax compromise

On Apr 2, 2014, at 2:06 AM, Stefan Håkansson LK wrote:

> Thanks a lot for working out this proposal!
> 
> I like it.
> 
> Just for confirmation: getCapabilities, getConstraints and getSettings 
> would work as today (no changes compared to the Ed's draft)?

getCapabilities and getSettings should be unchanged.  I'll have to think about getConstraints, but off the top of my head it will either be able to remain the same or it will return this new structure.

> 
> Another thing, the sourceId "constraint" for use with getUserMedia was 
> discussed a bit the last week. But it can be used with this structure, 
> right?

I'll have to think about this one, but off the top of my head it should also work.

> 
> A couple of comments inline.
> 
> Stefan
> 
> 
> 
> 
> On 2014-04-02 03:19, Dan Burnett wrote:
>> At the teleconference last week we were tasked with finding a way to
>> adjust the current constraint syntax in the specification, without loss
>> of expressivity, into a syntax that was WebIDL-compatible.  We may have
>> done one better.
>> 
>> We have found a workable compromise.  It can be expressed with
>> dictionariesin WebIDL, retains the full expressive power of the current
>> proposal, and offers simpler default operation to users. Here's
>> a rich example of the syntax to study (full WebIDL below):
>> 
>>   var constraints =
>>   {
>>     require: ["width", "height"],
>>     width: {min: 640},
>>     height: {min: 480},
>>     frameRate: 40,
>>     facingMode: "user",
>>     advanced: [{width: 1920, height: 1280},
>>                {aspectRatio: 1.3333333333}]
>>   };
>> 
>> In this dictionary, there are three types of constraints: required,
>> non-required and advanced (you wouldn't use all three typically):
>> 
>> 1. Required.
>> 
>>    Members of the top-level dictionary whose names are in the 'require'
>>    list (the first 'width' and 'height' members in the example.)
>>    Collectively, these correspond to the mandatory ConstraintSet in the
>>    existing spec.  The UA must fail if any top-level members named in
>>    the require list are missing or unsupported by the UA.  Given how
>>    dictionaries work, this means that required constraints unknown
>>    to the UA fail.
>> 
>> 2. Non-required.
>> 
>>    Remaining members of the top-level dictionary whose names are NOT in
>>    the 'require' list  (the top-level 'frameRate' and 'facingMode'
>>    entries in the example.) These do not correspond to anything in the
>>    existing spec.
> 
> This would basically be "hints"? I think that in the spec, the first 
> example should only use non-required (as a simple start where the app hints

Yes, except that they are actually constraints rather than hints.  If you'll notice in the description below, there would be a requirement to satisfy as many as possible.  This means that a UA must use/apply the non-required constraints given when possible and as specified, resulting in a clear determination for the UI of how it is to develop its recommended device selection or configuration.  It cannot arbitrarily choose to ignore non-required constraints that would be in the most-satisfiable set.
As with Firefox it may be possible for the user to override the recommended device selection as long as it does not violate the required constraints, but that is an orthogonal decision, since the UA is still required to follow the given selection/application algorithm in determining its recommendation.

Regarding examples, it will probably be useful to show increasing complexity, i.e.,
- an example with only non-required constraints
- one with required and non-required constraints
- one with required and advanced constraints
- one with all 3 kinds

> 
>> 
>> 3. Advanced.
>> 
>>    Dictionaries sequenced in 'advanced'. These correspond to the
>>    optional sequence of ConstraintSets in the existing spec.
> 
> And, I take it, you can repeat an entry several times here (as in the 
> current optional sequence?

Yes, this array is exactly the same as the current optional sequence.  Min and Max values are allowed as well, even though they're not shown in the example.
> 
>> 
>> 
>> Any of these three types of constraints may be absent from a call to
>> getUserMedia  or track.applyConstraints. If all three are missing, then
>> the call is unconstrained, which in the case of gUM means the UA can
>> pick any device it wants, and in the case of applyConstraints means all
>> currently applied constraints must be removed.
>> 
>> gUM works substantially as it does now, with the addition that the new
>> non-required constraints must be applied as a final step.  Specifically,
>> when gUM is called, the UA must first remove from consideration all
>> devices that do not meet all of the required constraints.  If there are
>> no devices left, the UA must call the error callback.  Otherwise, the
>> next step is as defined in the current spec, that it must iterate over
>> the optional (now advanced) ConstraintSets to perform a ranking, at each
>> round removing from a so-called "winners' pool" all devices that do not
>> satisfy the current ConstraintSet.  If in any round there are no devices
>> left, the UA must back up to the set from the previous round and skip to
>> the next round.  When the UA ends this process, it must finally rank
>> just the winners by how many individual non-required constraints they
>> satisfy.  Non-required constraints are to be considered individually and
>> order must not matter.  If this still does not produce a winner, the UA
>> must decide using any tie-breaker it wants.
>> 
>> When applyConstraints is called, if the Track in question cannot satisfy
>> all the required constraints, the UA must call the failure callback.
>> Otherwise, it must apply the required constraints, then apply the
>> 'advanced' ConstraintSets in order, skipping any ConstraintSet that
>> cannot be satisfied.  When it finishes, the UA must attempt to apply,
>> individually, each non-required constraint.  It must satisfy the largest
>> number of non-required constraints it can, in any order. Then the UA
>> must call the success callback.
> 
> And overconstrained would fire if any of the required constraints can no 
> longer be satisfied?

Yes.

> 
>> 
>> WebIDL (tentative):
>> dictionary MediaStreamConstraints {
>>     (boolean or MediaTrackConstraints) video = false;
>>     (boolean or MediaTrackConstraints) audio = false;
>>     DOMString peerIdentity;
>> };
>> 
>> dictionary MediaTrackConstraints : MediaTrackConstraintSet {
>>     sequence<DOMString> require;
>>     sequence<MediaTrackConstraintSet> advanced;
>> };
>> 
>> dictionary MediaTrackConstraintSet {
>>     ConstrainLong width;
>>     ConstrainLong height;
>>     ConstrainDouble aspectRatio;
>>     ConstrainDouble frameRate;
>>     ConstrainVideoFacingMode facingMode;
>>     ConstrainDouble volume;
>>     ConstrainLong sampleRate;
>>     ConstrainLong sampleSize;
>>     boolean echoCancelation;
>>     ConstrainDOMString sourceId;
>> 
>>     // Extended through gUM-specific IANA registry.
>> };
>> 
>> typedef (long or ConstrainLongRange) ConstrainLong;
>> typedef (double or ConstrainDoubleRange) ConstrainDouble;
>> typedef (VideoFacingModeEnum or sequence<VideoFacingModeEnum>)
>> ConstrainVideoFacingMode;
>> typedef (DOMString or sequence<DOMString>) ConstrainDOMString;
>> 
>> 
>> 
>> Thanks,
>> 
>> Dan Burnett, Jan-Ivar Bruaroey, and Jim Barnett
>> 
> 
>

Received on Wednesday, 2 April 2014 11:49:17 UTC