Re: Spec question: Using settings dictionaries instead of MediaConstraints

On 06/21/2012 08:15 PM, Li Li wrote:
>
> Harald,
>
> I don’t have a concrete example for SessionDescription, but here is an 
> example for getUserMedia [1] to illustrate the problem.
>
> Suppose I want to select media tracks that satisfy either C1 or C2:
>
> C1:
>
> video-max-width:600,
>
> video-max-aspectratio:1.333333333333,
>
> video-min-framerate:24
>
> Or these:
>
> C2:
>
> video-min-width:1024,
>
> video-max-aspectratio:1.77777777777,
>
> video-min-framerate:60
>

In other words: You want low quality tracks in 4:3, OR high quality 
tracks in 16:9, you don't care about which one you get for any given 
call to getUserMedia, but you want one or the other.

What usage scenario do you imagine where you would want that?

If your app wants a high quality track and a low quality track, wouldn't 
it be more natural to just grab the high quality track and then the low 
quality track?

I'm not saying that the framework specified can do everything - it was 
not supposed to.
But generality is very costly.

> In current mandatory/optional constraint framework, I can’t merge C1 
> and C2 into the mandatory part as no tracks will satisfy both. If I 
> put C1 before C2 in the optional part, I may get an empty set if there 
> are tracks that satisfy C2 but not C1. This is because the selection 
> algorithm will not try C2 if C1 fails.
>
> Similar problem may occur if I put C2 before C1 in the optional part.
>
> It seems a more flexible way to represent constraints is to allow OR 
> of an array of AND constraints.
>
> If C1 and C2 are JS objects of primitive constraints, the above 
> constraint can be represented as a JS array [C1, C2], interpreted as 
> “C1 OR C2”.
>
> Any mandatory/optional split could also be represented as such:
>
> var m = {….}; // the primitive mandatory constraints
>
> var o = {…}; // the primitive optional constraints
>
> var c = [[m, o], m]; // interpreted as “(m AND o) OR m”
>
> Please correct me if I misunderstood the current constraint framework.
>
> Thanks.
>
> Li
>
> [1] 
> http://lists.w3.org/Archives/Public/public-media-capture/2012Mar/0055.html.
>
> *From:*Harald Alvestrand [mailto:harald@alvestrand.no]
> *Sent:* Thursday, June 21, 2012 1:22 AM
> *To:* Li Li
> *Cc:* Justin Uberti; public-webrtc@w3.org
> *Subject:* Re: Spec question: Using settings dictionaries instead of 
> MediaConstraints
>
> On 06/21/2012 12:22 AM, Li Li wrote:
>
> I have a question about the optional constraints. If either optional 
> constraint C1 or C2 is OK, how would I express this choice?
>
> In current syntax, do I have to create two constraint groups: G1={C1, 
> C2} and G2={C2, C1}, and call pc.createOffer(f1, f2, G2) if 
> pc.createOffer(f1, f2, G1) fails?
>
>
> I'm afraid I can't parse this in the abstract. One of the desires with 
> the constraint syntax (which should really be discussed on the media 
> TF list) was to make it less complex than Yet Another Programming 
> Language; one thing it doesn't do is conditionals.
>
> Could you give a real world example of a case where you have an 
> optional constraint with two possible (ranges of?) values, where both 
> are OK, but some other values are not OK?
>
>
>
>
> Thanks.
>
> Li
>
> *From:*Justin Uberti [mailto:juberti@google.com]
> *Sent:* Wednesday, June 20, 2012 4:04 PM
> *To:* Harald Alvestrand
> *Cc:* public-webrtc@w3.org <mailto:public-webrtc@w3.org>
> *Subject:* Re: Spec question: Using settings dictionaries instead of 
> MediaConstraints
>
> On Wed, Jun 20, 2012 at 11:11 AM, Harald Alvestrand 
> <harald@alvestrand.no <mailto:harald@alvestrand.no>> wrote:
>
> On 06/20/2012 04:56 PM, Justin Uberti wrote:
>
> On Wed, Jun 20, 2012 at 9:21 AM, Harald Alvestrand 
> <harald@alvestrand.no <mailto:harald@alvestrand.no>> wrote:
>
> On 06/19/2012 03:45 PM, Justin Uberti wrote:
>
> On Tue, Jun 19, 2012 at 8:57 AM, Stefan Hakansson LK 
> <stefan.lk.hakansson@ericsson.com 
> <mailto:stefan.lk.hakansson@ericsson.com>> wrote:
>
> On 06/19/2012 08:30 AM, Randell Jesup wrote:
>
> On 6/18/2012 3:22 PM, Justin Uberti wrote:
>
>
>
> On Mon, Jun 18, 2012 at 2:57 PM, Cullen Jennings (fluffy)
> <fluffy@cisco.com <mailto:fluffy@cisco.com><mailto:fluffy@cisco.com 
> <mailto:fluffy@cisco.com>>>  wrote:
>
>
>     This seems like good proposal, one comment on a small detail.
>
>     On Jun 15, 2012, at 1:28 PM, Justin Uberti wrote:
>
> >  SessionDescriptionOptions.IncludeAudio = true/false // forces
>     m=audio line to be included
> >  SessionDescriptionOptions.IncludeVideo = true/false // forces
>     m=video line to be included
> >  SessionDescriptionOptions.UseVoiceActivityDetection = true/false
>     // includes CN codecs if true
>
>     I think these three should be constraints, not settings because a
>     given browser may not support any of them.
>
>
> Practically speaking, what does that mean for applications?
>
>
> I can conceive of a browser implementing audio but not video.  And a
> gateway or other stand-alone WebRTC box/functionality might include JS
> and these JS apis for ease of programming (and might be audio-only).
> (I'd try to avoid it in production, probably, but even that might not be
> needed with modern JS JIT speed so long as it didn't have to tear down
> and restart all the time.)
>
> CN codecs: I dislike them anyways.  :-)  An implementation definitely
> could avoid including those.
>
> Many codecs have built in CN modes. I guess for those it is more a 
> question of being able to switch off the VAD.
>
> I agree with the scenarios mentioned - my question was mostly about 
> what does having these settings be "constraints" vs "settings" mean 
> for users of the API. Is it simply that the call must fail if the 
> request can't be satisfied?
>
> If they are constraints, it's the caller's choice whether or not the 
> call should fail if the constraint can't be satisfied (listing them in 
> "mandatory" vs "optional" sections).
>
> If they are settings, it's the specification's choice whether or not 
> the call should fail if the constraint can't be satisfied; it has to 
> be always handled the same.
>
> At least that's how I read it.
>
> That makes sense. So perhaps we add .mandatory and .optional to make 
> these dictionaries into SessionDescriptionConstraints/IceConstraints, e.g.
>
> constraints = {};
>
> constraints.mandatory.getCapabilities = true;
>
> constraints.optional.includeAudio = true;
>
> constraints.optional.includeVideo = true;
>
> pc.createOffer(successCb, failCb, constraints);
>
> and we end up with MediaConstraints, SessionDescriptionConstraints, 
> and IceConstraints that all act the same way.
>
> Now the only difference between these constraints (?) and the 
> constraints defined in the getUserMedia draft is the initializer 
> syntax and the fact that we can't say that some optional constraints 
> are more important than others.
>
> If we're that close, I'd recommend going with the 
> getUserMedia-specified syntax.
>
> Let's just do one.
>
> That's pretty much what I had in mind, I didn't realize that 
> constraints already had ordering for optional items.
>
> The question becomes how do we best indicate which constraints can be 
> passed to the various API calls - I was thinking we can have different 
> types of XXXXConstraints for different API calls, and we can indicate 
> what values are valid for each type of XXXXConstraints where we 
> declare the type.
>
> Open to other ideas too though.
>