W3C home > Mailing lists > Public > public-media-capture@w3.org > October 2012

RE: Settings API Proposal v3 feedback summary

From: Travis Leithead <travis.leithead@microsoft.com>
Date: Fri, 5 Oct 2012 23:08:01 +0000
To: Stefan Hakansson LK <stefan.lk.hakansson@ericsson.com>
CC: "public-media-capture@w3.org" <public-media-capture@w3.org>
Message-ID: <9768D477C67135458BF978A45BCF9B3853A4C781@TK5EX14MBXW605.wingroup.windeploy.ntdev.microsoft.com>
> From: Stefan Hakansson LK [mailto:stefan.lk.hakansson@ericsson.com]
> On 10/05/2012 02:11 AM, Travis Leithead wrote:
> >> From: Stefan Hakansson LK
> >> [mailto:stefan.lk.hakansson@ericsson.com]
> >>
> >> Travis,
> >>
> >> many thanks for putting this together. I think that overall I like
> >> this a lot.
> >
> > After reading through this, what I didn't get around to was the
> > examples. I think I'll need to create some, since code samples are a
> > bit easier to follow than interface definitions :)
> Examples are always good to have :)

I added a few in a new section 6 in the proposal.

> >> Just to make sure I got the main things right, my understanding
> >> is:
> >>
> >> * the getUserMedia does not deliver a LocalMediaStream any more,
> >> instead it is more of a "gate", which once opened leads to the UA
> >> building up "DeviceListAccess" lists
> >
> > Not sure you got it... getUserMedia works almost the same as before,
> > but delivers a generic MediaStream (instead of a LocalMediaStream).
> > It's the tracks in the MediaStream that are fundamentally different
> > (extended).
> Aha, I missed that part. But would it work like this them:
> * The app calls getUserMedia; and as a result delivers a MediaStream
> with, say, one video and one audio track (the devices the user selected)
> * But another result is that device lists are created, one audio device
> list and one video device list; these lists can contain more devices -
> so the app would know more cams/mikes are available
> * Q: are all relevant devices in these lists, or just the ones that
> passed the mandatory constrains? (I would guess all of them)

All of the devices that are available to the application (not just the 
ones that met the constraints). Note, that some devices that may be in
use by other applications are not in the list, but could join the list
once they are available (and there's an event for that).

> * Q: would the app need to call getUserMedia to get access to one of
> them? (I think it should have to)

My current thinking is that the app would not need to call getUserMedia
to get access to one of the other devices in a given device list.
getUserMedia is not really equipped to be able to "grant access" to an
existing known device--it's more of a black-box request mechanism. So, 
for example, you can't have a device X, and pass it to getUserMedia for 
permission. (Well, I guess you could, but we'd have to alter getUserMedia
for that scenario.) The bigger question is what's the risk to allowing
access to all the available video devices or audio devices at once?
The biggest risk I see is that the app surreptitiously turns on the 
front-facing camera after you approve the back-facing camera and begins
"spying" on you (or vice-versa to spy on your surroundings). There's a
valid concern here, one we can probably mitigate if necessary.

> All in all, I'm starting to like this model more and more.


> >> My conclusion thus far: I think it is fine that the app has to
> >> construct a MediaStream from device tracks (rather than getting a
> >> LocalMediaStream that is ready to use). It is a bit more
> >> cumbersome, but not that much. I'm a bit more hesitant to that new
> >> devices just populate the "DeviceListAccess" and can be used. I
> >> have a couple of issues with this:
> >>
> >> 1. Even if the user is OK with video from a cam facing one way it
> >> is not certain that she/he would be OK with sharing another view
> >> (and I think the discussion in WhatWG ages ago said that the user
> >> should have to manually select the devices that the app could use
> >> just like for the File API to avoid accidental leak of privacy)
> >
> > If we as a WG still feel this way, the proposal can be simplified to
> > only reveal the count of devices available, for example, rather than
> > a device list.
> And that the app would have to do another getUserMedia call to get
> access to one or more of the other devices? I think that would make sense.

I wonder is the problem "turning on the device w/out access" or is the problem
being able to inspect settings, etc.? If the problem is just about turning on the
device, we might be able to come up with a creative solution as noted earlier, 
while keeping the advantages of the DeviceList.

> >
> >
> >> 2. For devices where the underlying system cannot resolve camera
> >> direction (front, back) the old model allowed the app to e.g. have
> >> a button "select a front facing cam" which when clicked issue
> >> "getUserMedia". This will be more difficult in your proposal if I
> >> understand correctly (I guess that the app could display the video
> >> of every cam in a small video element and ask the user to click on
> >> the right one though - so this may be a minor issue, except for:)
> >
> > Actually, the easiest way to do this, would be to either 1) use an
> > optional constraint in getUserMedia (which is still possible with
> > this proposal), or 2) get the device list and call select() passing
> > in that constraint, and the resulting device object (if any) will be
> > given in the event handler. (select() is pretty much exactly like
> > getUserMedia's constraint system, it just provides results in an
> > event, instead of returning and activating a MediaStream.)
> I meant for the case when the system has no idea of where cameras are
> facing (like on my computer when I plug in a USB camera). The only way
> to get the right one in those cases is to involve the user (and perhaps
> a selfview).

One of the example uses of the proposal (that I just added) is to build
a self-view for all the devices. So, even if the UA didn't provide a very
good getUserMedia permissions experience, the app can still help the user
select the right device.

> >> 3. When will the devices in the "DeviceListAccess" be reserved
> >> (stopping other web or native apps from using them)? When they are
> >> added to the list or when they are used? There was a clear model
> >> for this with getUserMedia. (And even if they are not reserved
> >> until used, there is the risk that the app uses all of them to
> >> enable the user to select the right one, and then never releases
> >> them). And when would htey be released?
> >
> > Good question. I'll need to think about this.
> If the app would have to do getUserMedia to be able to use them, then I
> think we have the model for this already.

Perhaps; on the other hand, you can just say (as I have in the proposal)
that if another application has an exclusive lock on the device and it
couldn't be enabled by getUserMedia even if one asked for it, then it won't
show up in the device list.
Received on Friday, 5 October 2012 23:08:33 UTC

This archive was generated by hypermail 2.4.0 : Friday, 17 January 2020 16:26:12 UTC