W3C home > Mailing lists > Public > public-media-capture@w3.org > March 2013

Re: Overconstrained

From: Adam Bergkvist <adam.bergkvist@ericsson.com>
Date: Thu, 28 Mar 2013 07:37:44 +0100
Message-ID: <5153E538.7030400@ericsson.com>
To: Martin Thomson <martin.thomson@gmail.com>
CC: "public-media-capture@w3.org" <public-media-capture@w3.org>
I pretty much agree with everything you're saying here. Together with 
the media types, "camera facing" is the single most important 
constraint. I think you bring up an important factor with the OS default 
selection as well.

On the topic of solving the few remaining cases, I just listed a few 
ways to get there. One that simply could be implemented in JS (ask 
again), another that would require some minor implementation (Harald's 
UI proposal), and a last one that would be us inventing something that 
requires a bit more implementation work. I think our current approach 
with constraints belongs to the third category.

So what do we really need? I would say the options to select "media 
type" and "camera facing". Let's have Harald's grant all cameras 
approach as well, and we can always implement the "ask again" approach 
in JavaScript. Should "media type" and "camera facing" be part of the 
constraints/settings concept? Perhaps not. It's not likely that you 
would like to change either media type or camera facing with the 
constraints/settings API later. We could just have a simple structure to 
represent these.


On 2013-03-27 18:20, Martin Thomson wrote:
> I agree with the sentiment here, but I'm not sure what conclusion you
> are driving toward.
> Here's my logical process on device selection:
> In a sizable proportion of cases, there will be no choice to make.
> There will be one source.
> In a large proportion of the remainder, the user will be the one
> deciding which input device to use.  Most likely, they will have done
> so in the OS or chrome beforehand.
> For users, the single most important factor in determining which
> camera to use will be where it is pointed.  Applications too.  In both
> cases, this will probably be to the exclusion of all other factors.
> In the vanishingly small subset of cases where the above is not
> sufficient, it is not a problem for an application to ask for another
> camera.  I'm willing to bet that those cases are going to be
> specialized applications with better-informed users that are
> sufficiently motivated and informed to comply.
> Facing therefore becomes the most important constraint.  The sourceId
> constraint allows for stability of selection over time, especially for
> those cases where a default doesn't work.
> You'll notice that this doesn't include anything on the key issue at
> hand.  That is, it says nothing about whether additional
> application-specified "preferences" are desirable.  That's because it
> largely doesn't factor into the decision-making process, except for
> those specialized applications, which I propose have an entirely
> different relationship with their users.
> That doesn't stop the browser or OS from promoting "better" input
> devices over "worse" ones so that applications get the most capable
> device when multiple options are present.  Clearly, that's just doing
> the right thing by users.  I just don't see any strong motivation for
> giving the application input to the selection process...for now.
> Harald's proposal to grant access to all devices by default is growing
> on me - it avoids a lot of the (theoretical) problems that optional
> constraints were invented to work around.  In fact, if this is the
> behaviour we get, then I become more firmly opposed to optional
> constraints.
> On 27 March 2013 00:55, Adam Bergkvist <adam.bergkvist@ericsson.com> wrote:
>> On 2013-03-26 17:14, Martin Thomson wrote:
>>> On 26 March 2013 03:45, Adam Bergkvist <adam.bergkvist@ericsson.com>
>>> wrote:
>>>> On 2013-03-25 18:35, Martin Thomson wrote:
>>>>> I would like to propose that we remove optional constraints.
>>>>> Sure, it's relatively easy to contrive examples, but I don't believe
>>>>> that there is anything that a real-world application would unable to
>>>>> do using a mandatory-only constraints API.
>>>> [...]  We
>>>> can't mimic the effect a second run of optional constraints would have on
>>>> those cameras unless we return both cameras to the app and run the
>>>> JS-version of the optional constraints as described above.
>>> This is exactly the sort of contrivance I was talking about.  Sure, we
>>> can prove, mathematically, that mandatory-only produces fewer points
>>> of articulation, but are you aware of a case where this results in a
>>> real problem?
>> I shouldn't have said "I'm not convinced because.." before the statement you
>> quoted above. What I meant to say that I think optional constraints are
>> completely replaceable one you have a MediaStream, but it's not entirely the
>> case at getUserMedia() time. Weather that's actually a problem in real life
>> is another question.
>> What I'm not really convinced about is "uninformed" mandatory constraints at
>> getUserMedia() time in general because I think it's easy to get it wrong and
>> shut out a lot of users from a service because of a badly formed mandatory
>> constraint.
>> Talking about real life situations.. I think in 90-something% of the cases
>> users will either have either one camera (laptop or desktop) or two cameras
>> where one camera is pointing at the user and the other pointing away from
>> the user (pad or phone). And in the pad/phone case, even though the camera
>> pointing away from the user has better specs, it's not that suitable for
>> conferencing. With that said, I think the camera direction is the single
>> most important constraint at getUserMedia() time. If the app has a camera
>> pointing in the right direction, it could do some tests on it and determine
>> that it's not good enough and tell the user about that in a nice way (and
>> even explain why the camera sucks).
>> So, the remaining problem is: there are more than one camera pointing in the
>> desired direction, the user is presented with a choice, and the user picks
>> the wrong one. Possible solutions to that problem (a few from a very large
>> set):
>> * Tell the user that the camera wasn't good enough and ask for the other
>> camera pointing in the same direction. We know there's one because the
>> SourceInfo sequence contains an other camera with the desired direction but
>> a different sourceId. We've also talked about revealing more information
>> when the user has granted permission to one camera; that would make the
>> second attempt fool-proof.
>> * Convince the user to to grant us access to all cameras pointing in the
>> desired direction (Harald's UI proposal). The app can now decide for it self
>> which camera to use, or inform the user that none of the cameras are good
>> enough. This may be less of a privacy issue than all cameras in *any*
>> direction, but may still not appeal to all.
>> * Have some way of informed filtering. This was already mentioned at the end
>> of the first bullet, but only after one device was granted. We could have a
>> specific question to get access to information about all cameras without
>> having to select a default camera just to get hold of that info. There are a
>> lot of ways to do this. One way is to prompt the user if it's ok to populate
>> the SourceInfo list with more information. An other way is to have a
>> callback (getUserMedia() argument?) that, on a request from the selector UI,
>> gets all candidate devices in a sequence, the app can then filter out all
>> undesired devices and return the filtered sequence. Once the selector UI is
>> presented, the user can choose to show all devices or the filtered list.
>> /Adam
Received on Thursday, 28 March 2013 06:39:15 UTC

This archive was generated by hypermail 2.4.0 : Friday, 17 January 2020 16:26:15 UTC