RE: Spec question: Using settings dictionaries instead of MediaConstraints

Harald,

I don’t have a concrete example for SessionDescription, but here is an example for getUserMedia [1] to illustrate the problem.

Suppose I want to select media tracks that satisfy either C1 or C2:

C1:
video-max-width:600,
video-max-aspectratio:1.333333333333,
video-min-framerate:24

Or these:

C2:
video-min-width:1024,
video-max-aspectratio:1.77777777777,
video-min-framerate:60

In current mandatory/optional constraint framework, I can’t merge C1 and C2 into the mandatory part as no tracks will satisfy both. If I put C1 before C2 in the optional part, I may get an empty set if there are tracks that satisfy C2 but not C1. This is because the selection algorithm will not try C2 if C1 fails.
Similar problem may occur if I put C2 before C1 in the optional part.

It seems a more flexible way to represent constraints is to allow OR of an array of AND constraints.

If C1 and C2 are JS objects of primitive constraints, the above constraint can be represented as a JS array [C1, C2], interpreted as “C1 OR C2”.

Any mandatory/optional split could also be represented as such:

var m = {….}; // the primitive mandatory constraints
var o = {…}; // the primitive optional constraints
var c = [[m, o], m]; // interpreted as “(m AND o) OR m”

Please correct me if I misunderstood the current constraint framework.
Thanks.
Li


[1] http://lists.w3.org/Archives/Public/public-media-capture/2012Mar/0055.html.


From: Harald Alvestrand [mailto:harald@alvestrand.no]
Sent: Thursday, June 21, 2012 1:22 AM
To: Li Li
Cc: Justin Uberti; public-webrtc@w3.org
Subject: Re: Spec question: Using settings dictionaries instead of MediaConstraints

On 06/21/2012 12:22 AM, Li Li wrote:
I have a question about the optional constraints. If either optional constraint C1 or C2 is OK, how would I express this choice?
In current syntax, do I have to create two constraint groups: G1={C1, C2} and G2={C2, C1}, and call pc.createOffer(f1, f2, G2) if pc.createOffer(f1, f2, G1) fails?

I'm afraid I can't parse this in the abstract. One of the desires with the constraint syntax (which should really be discussed on the media TF list) was to make it less complex than Yet Another Programming Language; one thing it doesn't do is conditionals.

Could you give a real world example of a case where you have an optional constraint with two possible (ranges of?) values, where both are OK, but some other values are not OK?





Thanks.
Li

From: Justin Uberti [mailto:juberti@google.com]
Sent: Wednesday, June 20, 2012 4:04 PM
To: Harald Alvestrand
Cc: public-webrtc@w3.org<mailto:public-webrtc@w3.org>
Subject: Re: Spec question: Using settings dictionaries instead of MediaConstraints


On Wed, Jun 20, 2012 at 11:11 AM, Harald Alvestrand <harald@alvestrand.no<mailto:harald@alvestrand.no>> wrote:
On 06/20/2012 04:56 PM, Justin Uberti wrote:

On Wed, Jun 20, 2012 at 9:21 AM, Harald Alvestrand <harald@alvestrand.no<mailto:harald@alvestrand.no>> wrote:
On 06/19/2012 03:45 PM, Justin Uberti wrote:

On Tue, Jun 19, 2012 at 8:57 AM, Stefan Hakansson LK <stefan.lk.hakansson@ericsson.com<mailto:stefan.lk.hakansson@ericsson.com>> wrote:
On 06/19/2012 08:30 AM, Randell Jesup wrote:
On 6/18/2012 3:22 PM, Justin Uberti wrote:


On Mon, Jun 18, 2012 at 2:57 PM, Cullen Jennings (fluffy)
<fluffy@cisco.com<mailto:fluffy@cisco.com><mailto:fluffy@cisco.com<mailto:fluffy@cisco.com>>>  wrote:


    This seems like good proposal, one comment on a small detail.

    On Jun 15, 2012, at 1:28 PM, Justin Uberti wrote:

     >  SessionDescriptionOptions.IncludeAudio = true/false // forces
    m=audio line to be included
     >  SessionDescriptionOptions.IncludeVideo = true/false // forces
    m=video line to be included
     >  SessionDescriptionOptions.UseVoiceActivityDetection = true/false
    // includes CN codecs if true

    I think these three should be constraints, not settings because a
    given browser may not support any of them.


Practically speaking, what does that mean for applications?

I can conceive of a browser implementing audio but not video.  And a
gateway or other stand-alone WebRTC box/functionality might include JS
and these JS apis for ease of programming (and might be audio-only).
(I'd try to avoid it in production, probably, but even that might not be
needed with modern JS JIT speed so long as it didn't have to tear down
and restart all the time.)

CN codecs: I dislike them anyways.  :-)  An implementation definitely
could avoid including those.

Many codecs have built in CN modes. I guess for those it is more a question of being able to switch off the VAD.

I agree with the scenarios mentioned - my question was mostly about what does having these settings be "constraints" vs "settings" mean for users of the API. Is it simply that the call must fail if the request can't be satisfied?

If they are constraints, it's the caller's choice whether or not the call should fail if the constraint can't be satisfied (listing them in "mandatory" vs "optional" sections).

If they are settings, it's the specification's choice whether or not the call should fail if the constraint can't be satisfied; it has to be always handled the same.

At least that's how I read it.

That makes sense. So perhaps we add .mandatory and .optional to make these dictionaries into SessionDescriptionConstraints/IceConstraints, e.g.

constraints = {};
constraints.mandatory.getCapabilities = true;
constraints.optional.includeAudio = true;
constraints.optional.includeVideo = true;
pc.createOffer(successCb, failCb, constraints);

and we end up with MediaConstraints, SessionDescriptionConstraints, and IceConstraints that all act the same way.
Now the only difference between these constraints (?) and the constraints defined in the getUserMedia draft is the initializer syntax and the fact that we can't say that some optional constraints are more important than others.

If we're that close, I'd recommend going with the getUserMedia-specified syntax.

Let's just do one.

That's pretty much what I had in mind, I didn't realize that constraints already had ordering for optional items.

The question becomes how do we best indicate which constraints can be passed to the various API calls - I was thinking we can have different types of XXXXConstraints for different API calls, and we can indicate what values are valid for each type of XXXXConstraints where we declare the type.

Open to other ideas too though.