RE: Constraints at request time (Re: Revised Constraints modification API proposal)

> From: Dan Burnett [mailto:dburnett@voxeo.com]
> 
> On Aug 23, 2012, at 9:50 AM, Adam Bergkvist wrote:
> 
> > On 2012-08-23 12:14, Harald Alvestrand wrote:
> >> Forking this thread.....
> >>
> >> On 08/22/2012 08:35 PM, Travis Leithead wrote:
> >>>> From: Dan Burnett [mailto:dburnett@voxeo.com] On Aug 21, 2012, at
> >>>> 4:56 PM, Travis Leithead wrote:
> >>>>> As a reminder, the goal of this proposal is to facilitate
> >>>>> "informed constraints" (i.e., allow constraints to be applied
> >>>>> after existing client capabilities are known) in order to avoid
> >>>>> potential pitfalls of blindly over-constrained use of getUserMedia
> >>>>> across a range of different
> >>>> devices.
> >>>>
> >>>> I always wanted to have capabilities along with constraints, and
> >>>> I'm pleased to see (finally) a realistic capabilities proposal.
> >>>> Given that we will not always have capabilities available for
> >>>> privacy reasons, I'd like to understand better these "potential
> >>>> pitfalls of blindly over-constrained use".  I have heard that
> >>>> mentioned but have not yet seen enough evidence to convince me
> that this is a real problem.  Could you point me to some examples?
> >>> [TODO]
> >>>
> >>>
> >>>>> A secondary goal is to provide the right set of APIs for uniformly
> >>>>> working with the devices that supply the local media stream
> >>>>> tracks, for future APIs and scenarios we may wish to add.
> >>>>>
> >>>>> If this proposal is adopted, I would expect that the existing
> >>>>> constraint usage in getUserMedia could be significantly scaled
> >>>>> back, if not removed altogether.
> >>>> With the device-level only focus below, I don't think that would quite
> work.
> >>>> One of the goals of the constraints approach was to free a
> >>>> developer who just wants "a camera" from having to worry about
> >>>> which one.  They just specify their constraints, mandatory or
> >>>> optional based on their needs, and if they get something it is
> something they can use.
> >>> I don't fully understand why the "device-level focus" prevents what
> >>> you said from working. Even with the spec-as-is, this only gives a
> >>> developer one-shot to get the right settings. For example, if they
> >>> specify a constraint for some camera setting (perhaps white balance
> >>> or flash?) and when they get a chance to look at the result stream
> >>> and don't like it... well then you would probably expect that there
> >>> would be another setting for white balance or flash on the track.
> >>> So, we're in a situation where we have a set of constraints, and a
> >>> set of settings which are basically duplicates. With the proposal below, it
> consolidates these concepts. Using the proposal below, I would expect:
> >>>
> >>> 1. User requests "video" from getUserMedia (because they want "a
> >>> camera") 2. If there's some device that provides video, it is provided as a
> track on a local
> >>>      media stream.
> >>> 3. Now, the developer applies the same constraints to that track, and if
> the
> >>>      contraints were applied, then we're good to go.
> >>>
> >>> Not too much more complicated than the current draft. If the
> >>> developer doesn't like the result, they can just re-apply a
> >>> different set of constraints to the same track, and see if that works for
> them. This IMO is much simpler.
> >> I still don't understand why it is simpler.
> >>
> >> If the UA knows what constraints the application desires, he can
> >> offer the user only the choice of cameras that satisfy the constraint.
> >>
> >> If the UA doesn't know, the dialog becomes:
> >>
> >> App: "UA, please give me a camera"
> >> UA: "User, Please give permission to access a camera"
> >> User: "OK, I give it camera 1"
> >> App: "User, I didn't like that camera, please try another"
> >> UA: "User, Please give permission to access a camera"
> >> User: "OK, I give it camera 2"
> >> App: "OK, that camera worked"
> >>
> >> Having an adequate user dialog to explain that the camera he picked
> >> the first time is unsuitable, and how to tell which camera is which
> >> so that the user doesn't pick it again, is left as an exercise for
> >> the application designer.
> >>
> >> With the ability to apply changed constraints, we can still do:
> >>
> >> UA: "User, please give me a camera" (some constraints apply)
> >> User: "OK, I give it camera 1"
> >> App: "User, does this picture look OK to you?"
> >> User: "No, let's try this tweak"
> >> App: "OK, tweak applied. Better?"
> >>
> >> I *like* applying changed constraints.
> >
> > I've also though about this initial filtering that applying constraints before
> presenting device options to the user provides. However, I think we win
> more with the new approach than we loose.
> >
> > Even with constraints applied after device selection we need some small
> set of options/constraints to filter the device list presented to the user. We
> have the obvious "audio" and "video", but we may also need something to
> hint about camera direction where it's possible ("front"/"user"
> "back"/"environment"). If this proves to be a huge problem, we could also
> look into some quality hints here like "hd", "vga" and so on. It won't solve
> everything but I think it will cover a lot of cases.
> 
> Let's please not go all the way back to hints.  We started there and ended up
> in a better place with constraints on getUserMedia.  If you want to know the
> reasons to go beyond hints, please re-read my post on that many months
> ago at http://lists.w3.org/Archives/Public/public-media-
> capture/2012Feb/0041.html.

Are optional-only constraints to getUserMedia the same as hints?

In the after-the-fact constraint modification API, the applied constraints would likely all be treated as required constraints (optional constraints really don't make sense at that point--e.g., you *really* want the settings to be applied) with some kind of failure notification.

Received on Thursday, 23 August 2012 19:11:41 UTC