Re: Proposal: Constraints as dictionaries

Thanks, Jan-Ivar, for putting this proposal together. I wonder how much 
of your brain capacity that has been consumed by constraints lately. :)

On 2013-11-20 22:30, Jan-Ivar Bruaroey wrote:
> I've been asked to make a concrete proposal to fix problems brought up
> with constraints. There are three parts (ABC).
>
>
>       A. Don't violate WebIDL
>
>
> Constraints must be WebIDL dictionaries. We cannot afford to reinvent
> dictionaries or webidl just for Constraints, at this point. The WebIDL
> WG have thought more about JS and JS processing models than I think we
> have, and for the complexity and long-tail use-cases inherent in this
> API, we need explicit definitive language that implementers can follow,
> and WebIDL is that language.
>
> WebIDL dictionaries ignore unknown keys. The following tweaks in our
> model let constraints be dictionaries:
>
>   1. Conservative programmer calls if
> (!getGumKnownConstraints().hasOwnProperty("3D")) then bail
>   2. getUserMedia({ mandatory: { 3D: true } }, succ, fail); // where 3D
> is ignored if unimplemented
>   3. (minor) Remove limit of one key-value pair in dictionaries in
> optional array (over-specified)
>   4. KeepConstrainable as a pattern, but root it in actual gUM types
> (remove abstractions, 'any' etc.)
>   5. Change abstract PropertyValueRange to UnsignedLongRange, FloatRange
> etc.
>
> Most controversial difference: The API for detecting browser-support for
> a constraint is now explicit, and gUM() itself only constrains by
> properties it knows.
>
> This avoids blanket blocking by browser for bad as well as good cameras,
> something simple apps may not have signed on for, and lets apps
> explicitly block access on those same browsers with one line of code.
>
> Not new: Code that plans to control the camera must already use
> getCapabilities() to learn the known things they can modify, so there is
> no added cost in those cases.
>
> We then describe Constraints, Capabilities and Settings in the spec
> using WebIDL instead of abstract prose. I circulated
> http://lists.w3.org/Archives/Public/public-media-capture/2013Nov/0084.htmlearlier
> which shows the proposed WebIDL at the bottom. Using WebIDL
> descriptively in this manner does not preclude IANA.

I think what you say here makes sense. Is the thinking that two optional 
constraints with the same priority (point 3.) should be evaluated 
together as "constraint 1" && "constraint 2"? Then again, we only need 
to think about it if we stick with the different types of constraints.

>       B. Don't leak
>
>
> Stop returning ConstraintNotSatisfiedError, because it leaks information
> about capabilities to the website without user consent, letting a
> malicious website paint a full picture after a few visits.
>
> Instead, always bring up the permission prompt, even when there is no
> match.The UA MUST warn the user differently of such non-matches and
> offer the Deny-equivalent choice only. The UA MUST NOT let the user
> grant access to a camera in this case (not because of a leak, but to
> give webpages an invariant for known properties).

We said on the TPAC meeting that we should make mandatory constraints 
"harder to use" since they are so easy to miss-use. However, the 
conclusion I come to when looking at our spec is that mandatory 
constraints are much easier to understand and use (compared to optional 
ones). With mandatory you get what you ask for (with some limitations), 
and if the browser no longer can sustain a mandatory constraint, you 
will be notified. The point of providing optional constraints is less 
obvious. They may be disregarded at any time and to UA won't tell you 
when doing so. Optional constraints do provide a feature to prefer high 
frame rate over resolution, but it's not that obvious how it's done. I 
might be missing something but will anyone even use optional constrains 
under these circumstances? This proposal doesn't make mandatory 
constraints harder to use, but it could make them a bit less exclusive.

With this proposal, even if you don't have a good enough device, you 
will at least get something, the prompt, even if the app has an empty 
error callback on getUserMedia() that ideally should have informed the 
user about the situation. It's now up to the browser to let users down, 
but at least it can be done consistently.

We've have, pretty much since day one, said that it should be possible 
to provide a file as a media source to not exclude users. (The 
conclusion at some discussion was that picking a file would be similar 
to connecting a new device.) By always prompting the user we have a 
better opportunity to offer that service, compared with the 
ConstraintNotSatisfiedError approach. This might be a crazy idea, but we 
could even offer a fake device that meets the requirements of the 
application.

>       C. New syntax
>
>
> A new syntax for Constraints only, is aimed at moving us away from the
> mandatory/optional language to a model focused on expressiveness, and
> away from Constraints being "things" to being dictionaries used in a
> pattern, just like Settings and Capabilities are now.
>
> In this syntax, Constraints is an array of Settings (dictionaries) in
> decreasing order of preference, where a set is accepted only if all its
> specified keys are. The first accepted set is chosen.
>
> To describe your camera preferences in fine detail, you provide all the
> combinations of settings you accept, specifying only the things you care
> about, in all the combinations you accept.
>
> I show a width/height example here
> http://lists.w3.org/Archives/Public/public-media-capture/2013Nov/0052.htmlwhich
> in turn has a link to the original post.
>
> Let me elaborate on the example in the original post (sans typo):
>> Consider an alternative to:
>>           { mandatory: { a, b }, optional: [ { c }, { d } ] }
>> like this:
>>         [ { a, b, c, d }, { a, b, c }, { a, b, d }, { a, b } ]
>> Old familiars and what they look like now:
>>      { mandatory: { a, b } } ==> [ { a, b } ]
>> and separately:
>>      { optional: [{ a }, { b }] } ==> [ { a }, { b }, {} ]
>
> Benefits of new syntax:
> + Simpler. No keyword-dependent structure variance ([{}] vs. {}).
> + Advantage of being able to combine related things like width + height
> naturally and logically.
> + {} at the end of the array naturally makes everything optional, since
> it means "I'll accept anything".
> + More power (allows for finer, more explicit selection).
> + (AND)OR array logic is easier to read than the current optional-array
> logic (esp. width/height combos)
> + Bye bye misunderstood mandatory keyword. Hello dictionaries. <3
>
> Cons:
> - Can't handle large number of optionals, grows exponentially in
> edge-cases. (6 = ugh, 30 = no way)
>    I'm banking on there always being patterns in what programmers desire
> (OR-ing sets not 1by1. Try it!)
> - Pilot error: Forget ending array with {} and everything is mandatory.
>
> Since we no longer tailor any data-types for our special needs, and rely
> solely on well-known constructs, how things work and don't work should
> be much clearer both to developers and programmers.

This looks reasonable, at least at first glance. :)

It's also a better fit with Constrainable.applyConstraints() since 
optional constraints don't really make sense in that case. I'm imagining 
feeding applyConstraints() with the same type as the elements in the 
array given to getUserMedia().

We should really get started on the document how constrains solves 
different use-cases. Obvious limitations will show up when the method 
face a real life situation.

/Adam

Received on Thursday, 21 November 2013 08:33:48 UTC