W3C home > Mailing lists > Public > public-media-capture@w3.org > October 2012

Re: Settings API Proposal v3 feedback summary

From: Stefan Hakansson LK <stefan.lk.hakansson@ericsson.com>
Date: Tue, 9 Oct 2012 14:52:27 +0200
Message-ID: <50741E0B.2070101@ericsson.com>
To: Travis Leithead <travis.leithead@microsoft.com>
CC: "public-media-capture@w3.org" <public-media-capture@w3.org>
On 10/06/2012 01:08 AM, Travis Leithead wrote:
>> From: Stefan Hakansson LK [mailto:stefan.lk.hakansson@ericsson.com]
>> On 10/05/2012 02:11 AM, Travis Leithead wrote:
>>>> From: Stefan Hakansson LK
>>>> [mailto:stefan.lk.hakansson@ericsson.com]
>>>> Travis,
>>>> many thanks for putting this together. I think that overall I like
>>>> this a lot.
>>> After reading through this, what I didn't get around to was the
>>> examples. I think I'll need to create some, since code samples are a
>>> bit easier to follow than interface definitions :)
>> Examples are always good to have :)
> I added a few in a new section 6 in the proposal.
>>>> Just to make sure I got the main things right, my understanding
>>>> is:
>>>> * the getUserMedia does not deliver a LocalMediaStream any more,
>>>> instead it is more of a "gate", which once opened leads to the UA
>>>> building up "DeviceListAccess" lists
>>> Not sure you got it... getUserMedia works almost the same as before,
>>> but delivers a generic MediaStream (instead of a LocalMediaStream).
>>> It's the tracks in the MediaStream that are fundamentally different
>>> (extended).
>> Aha, I missed that part. But would it work like this them:
>> * The app calls getUserMedia; and as a result delivers a MediaStream
>> with, say, one video and one audio track (the devices the user selected)
>> * But another result is that device lists are created, one audio device
>> list and one video device list; these lists can contain more devices -
>> so the app would know more cams/mikes are available
>> * Q: are all relevant devices in these lists, or just the ones that
>> passed the mandatory constrains? (I would guess all of them)
> All of the devices that are available to the application (not just the
> ones that met the constraints). Note, that some devices that may be in
> use by other applications are not in the list, but could join the list
> once they are available (and there's an event for that).
>> * Q: would the app need to call getUserMedia to get access to one of
>> them? (I think it should have to)
> My current thinking is that the app would not need to call getUserMedia
> to get access to one of the other devices in a given device list.
> getUserMedia is not really equipped to be able to "grant access" to an
> existing known device--it's more of a black-box request mechanism. So,
> for example, you can't have a device X, and pass it to getUserMedia for
> permission. (Well, I guess you could, but we'd have to alter getUserMedia
> for that scenario.) The bigger question is what's the risk to allowing
> access to all the available video devices or audio devices at once?
> The biggest risk I see is that the app surreptitiously turns on the
> front-facing camera after you approve the back-facing camera and begins
> "spying" on you (or vice-versa to spy on your surroundings). There's a
> valid concern here, one we can probably mitigate if necessary.

This is a privacy and user interaction question that I am very badly 
positioned to have a view on. But I note that the original WhatWg 
proposal had a model where no device were supposed to be "default" in 
the gUM dialog - because that could lead to a click-through behavior. I 
also got a comment from a friend that I had to test webrtc in Chrome the 
other day that he closed the browser and re-booted after trying it out 
because he did not know what devices, and for how long, the browser had 
access to - so I think these may be valid concerns.

So I think we may want a model were the UA can't give the app access to 
one more camera or mike without a user grant (even if they enter the 
device list). Exactly how we do that is another question.

>> All in all, I'm starting to like this model more and more.
> :-)
>>>> My conclusion thus far: I think it is fine that the app has to
>>>> construct a MediaStream from device tracks (rather than getting a
>>>> LocalMediaStream that is ready to use). It is a bit more
>>>> cumbersome, but not that much. I'm a bit more hesitant to that new
>>>> devices just populate the "DeviceListAccess" and can be used. I
>>>> have a couple of issues with this:
>>>> 1. Even if the user is OK with video from a cam facing one way it
>>>> is not certain that she/he would be OK with sharing another view
>>>> (and I think the discussion in WhatWG ages ago said that the user
>>>> should have to manually select the devices that the app could use
>>>> just like for the File API to avoid accidental leak of privacy)
>>> If we as a WG still feel this way, the proposal can be simplified to
>>> only reveal the count of devices available, for example, rather than
>>> a device list.
>> And that the app would have to do another getUserMedia call to get
>> access to one or more of the other devices? I think that would make sense.
> I wonder is the problem "turning on the device w/out access" or is the problem
> being able to inspect settings, etc.? If the problem is just about turning on the
> device, we might be able to come up with a creative solution as noted earlier,
> while keeping the advantages of the DeviceList.
>>>> 2. For devices where the underlying system cannot resolve camera
>>>> direction (front, back) the old model allowed the app to e.g. have
>>>> a button "select a front facing cam" which when clicked issue
>>>> "getUserMedia". This will be more difficult in your proposal if I
>>>> understand correctly (I guess that the app could display the video
>>>> of every cam in a small video element and ask the user to click on
>>>> the right one though - so this may be a minor issue, except for:)
>>> Actually, the easiest way to do this, would be to either 1) use an
>>> optional constraint in getUserMedia (which is still possible with
>>> this proposal), or 2) get the device list and call select() passing
>>> in that constraint, and the resulting device object (if any) will be
>>> given in the event handler. (select() is pretty much exactly like
>>> getUserMedia's constraint system, it just provides results in an
>>> event, instead of returning and activating a MediaStream.)
>> I meant for the case when the system has no idea of where cameras are
>> facing (like on my computer when I plug in a USB camera). The only way
>> to get the right one in those cases is to involve the user (and perhaps
>> a selfview).
> One of the example uses of the proposal (that I just added) is to build
> a self-view for all the devices. So, even if the UA didn't provide a very
> good getUserMedia permissions experience, the app can still help the user
> select the right device.

Yes, that can very well be done. Let's just hope that the app releases 
the devices that are not used!

>>>> 3. When will the devices in the "DeviceListAccess" be reserved
>>>> (stopping other web or native apps from using them)? When they are
>>>> added to the list or when they are used? There was a clear model
>>>> for this with getUserMedia. (And even if they are not reserved
>>>> until used, there is the risk that the app uses all of them to
>>>> enable the user to select the right one, and then never releases
>>>> them). And when would htey be released?
>>> Good question. I'll need to think about this.
>> If the app would have to do getUserMedia to be able to use them, then I
>> think we have the model for this already.
> Perhaps; on the other hand, you can just say (as I have in the proposal)
> that if another application has an exclusive lock on the device and it
> couldn't be enabled by getUserMedia even if one asked for it, then it won't
> show up in the device list.

You're right, this model is probably better.

Received on Tuesday, 9 October 2012 12:52:52 UTC

This archive was generated by hypermail 2.3.1 : Tuesday, 6 January 2015 21:24:37 UTC