Re: Settings API Proposal v3 feedback summary

On 10/05/2012 02:11 AM, Travis Leithead wrote:
>> From: Stefan Hakansson LK
>> [mailto:stefan.lk.hakansson@ericsson.com]
>>
>> Travis,
>>
>> many thanks for putting this together. I think that overall I like
>> this a lot.
>
> After reading through this, what I didn't get around to was the
> examples. I think I'll need to create some, since code samples are a
> bit easier to follow than interface definitions :)

Examples are always good to have :)

>
>
>> Just to make sure I got the main things right, my understanding
>> is:
>>
>> * the getUserMedia does not deliver a LocalMediaStream any more,
>> instead it is more of a "gate", which once opened leads to the UA
>> building up "DeviceListAccess" lists
>
> Not sure you got it... getUserMedia works almost the same as before,
> but delivers a generic MediaStream (instead of a LocalMediaStream).
> It's the tracks in the MediaStream that are fundamentally different
> (extended).

Aha, I missed that part. But would it work like this them:

* The app calls getUserMedia; and as a result delivers a MediaStream 
with, say, one video and one audio track (the devices the user selected)
* But another result is that device lists are created, one audio device 
list and one video device list; these lists can contain more devices - 
so the app would know more cams/mikes are available
* Q: are all relevant devices in these lists, or just the ones that 
passed the mandatory constrains? (I would guess all of them)
* Q: would the app need to call getUserMedia to get access to one of 
them? (I think it should have to)

All in all, I'm starting to like this model more and more.


>
>
>> * these lists are "live", devices can come and go, the user more
>> approves the use of video or/and audio and not specific devices
>
> If referring to the device lists, then yes, that is the intention.
> The user approves the "class" of device, rather than a specific
> device, since once one device of a given class has been delivered,
> all the other related devices can be easily discovered.
>
>> * to use a device, the app would have to use the MediaStream
>> constructor to build a MediaStream (which can then be consumed by a
>> video element, a PeerConnection, ...) from the devices in the
>> DeviceListAccess list
>
> No, in the simple case, the MediaStream returned by getUserMedia can
> be used directly with a video element, etc., as we've discussed in
> the past.

Right, I misunderstood that part.

>
>
>> My conclusion thus far: I think it is fine that the app has to
>> construct a MediaStream from device tracks (rather than getting a
>> LocalMediaStream that is ready to use). It is a bit more
>> cumbersome, but not that much. I'm a bit more hesitant to that new
>> devices just populate the "DeviceListAccess" and can be used. I
>> have a couple of issues with this:
>>
>> 1. Even if the user is OK with video from a cam facing one way it
>> is not certain that she/he would be OK with sharing another view
>> (and I think the discussion in WhatWG ages ago said that the user
>> should have to manually select the devices that the app could use
>> just like for the File API to avoid accidental leak of privacy)
>
> If we as a WG still feel this way, the proposal can be simplified to
> only reveal the count of devices available, for example, rather than
> a device list.

And that the app would have to do another getUserMedia call to get 
access to one or more of the other devices? I think that would make sense.

>
>
>> 2. For devices where the underlying system cannot resolve camera
>> direction (front, back) the old model allowed the app to e.g. have
>> a button "select a front facing cam" which when clicked issue
>> "getUserMedia". This will be more difficult in your proposal if I
>> understand correctly (I guess that the app could display the video
>> of every cam in a small video element and ask the user to click on
>> the right one though - so this may be a minor issue, except for:)
>
> Actually, the easiest way to do this, would be to either 1) use an
> optional constraint in getUserMedia (which is still possible with
> this proposal), or 2) get the device list and call select() passing
> in that constraint, and the resulting device object (if any) will be
> given in the event handler. (select() is pretty much exactly like
> getUserMedia's constraint system, it just provides results in an
> event, instead of returning and activating a MediaStream.)

I meant for the case when the system has no idea of where cameras are 
facing (like on my computer when I plug in a USB camera). The only way 
to get the right one in those cases is to involve the user (and perhaps 
a selfview).

>
>
>> 3. When will the devices in the "DeviceListAccess" be reserved
>> (stopping other web or native apps from using them)? When they are
>> added to the list or when they are used? There was a clear model
>> for this with getUserMedia. (And even if they are not reserved
>> until used, there is the risk that the app uses all of them to
>> enable the user to select the right one, and then never releases
>> them). And when would htey be released?
>
> Good question. I'll need to think about this.

If the app would have to do getUserMedia to be able to use them, then I 
think we have the model for this already.

>
>
>> -------------------------------------------------------------------
>>
>>
>>
Apart from those issues, that perhaps we can find a way around, I like
>> the proposal a lot. Some minor comments:
>>
>> Section 1.1: As already stated, I'm not sure that the user giving
>> OK to video is the same as allowing any camera to be used. I guess
>> this is a privacy and usability question that needs to be discussed
>> more.
>>
>> Section 2.1.1: Should not "frameRate" also be (readonly) attribute
>> for VideoStreamTrack?
>
> I thought about this--remote media coming from a peer connection
> might have a variable rate, and as such you'd want a more complete
> API for dealing with that. To keep things simple, I limited it to a
> video device track only.
>
>> Regarding the Setting/retrieval application, my understanding is
>> that a request would lead to that the constraints for the device in
>> question are changed. Correct? Does this also mean that the
>> constrainterror/success events could be fired as a result?
>
> Yes, those events should be fired in those cases.
>
>>
>> Stefan
>>
>>
>>
>>>
>>>
>>>> -----Original Message----- From: Travis Leithead Sent: Monday,
>>>> October 1, 2012 4:37 PM To: public-media-capture@w3.org
>>>> Subject: Settings API Proposal v3 feedback summary
>>>>
>>>> Before preparing the 4th version of the settings API proposal,
>>>> I
>> thought
>>>> it would be nice to step back and summarize the feedback
>>>> received on
>> v3.
>>>> I've not been as responsive to responding to individual
>>>> threads, so I'm using this one as an opportunity to aggregate
>>>> all the feedback into one location.
>>>>
>>>> There was lots of positive responses. Thank you!
>>>>
>>>> I have a v4 that I'll be releasing shortly, so stay tuned.
>>>>
>>>> -Travis
>>>>
>>>> 1  Liked the approach/heading in the right direction [Rich-2]
>>>> [Stefan-
>> 1]
>>>> [Adam-1] 2  Are all the various dictionaries values read/write
>>>> or read-only? [Stefan-1] 3  Should the constraints/settings be
>>>> inlined in the spec? (No strong preference) [Stefan-1] 4  Is
>>>> this proposal and constraints applied to gUM complimentary?
>> [Stefan-
>>>> 1] 5  Establishes a nice "source-of-track" pointer [Randell-1]
>>>> 6  Derivative of an earlier proposal (suggested in
>>>> [Randell-2]?) 7  Doesn't solve multiple consumers competing for
>>>> changing settings [Randell-1] (clarified by [Stefan-2/3]) 8
>>>> Forces all tracks that want changes to have a MediaDevice
>>>> (PeerConnection, Video element src) [Randell-1] [Stefan-2] 9
>>>> Proposal for unified device/track concept (duck-typing)
>>>> [Rich-2] (comments [Travis-2]) 10 Liked the prior proposal with
>>>> immutable track lists over the
>> devices-
>>>> based LMS [Adam-1] [Adam-3] 11 takePicture(); how is it related
>>>> to regular media stream recording? [Adam-1] 12 Can the *Info
>>>> dictionaries be exposed to gUM for early device filtering?
>>>> [Adam-1] 13 Should align with PeerConnection track ideas/
>>>> track-subtypes [Adam-
>> 2]
>>>> 14 Additional settings requested [Giridhar-1] 15 Proposal on
>>>> how to scope what settings we expose [Rich-1] 16 Idea on how to
>>>> consolidate/group the settings [Josh-1/2] [Jim-1] [Travis-1] 17
>>>> How do we define interoperable settings (how do we test them)?
>>>> [Frederick-1] [Jim-2] 18 How should UA's optionally surface
>>>> device-specific settings? [Jim-2] 19 Re-consider one
>>>> track-per-type in gUM? [Adam-3] 20 Don't hide capabilities
>>>> behind get/changeSettings APIs [Rich-2] (comments [Travis-2])
>>>>
>>>> [Stefan-1] http://lists.w3.org/Archives/Public/public-media-
>>>> capture/2012Sep/0000.html [Stefan-2]
>>>> http://lists.w3.org/Archives/Public/public-media-
>>>> capture/2012Sep/0002.html [Stefan-3]
>>>> http://lists.w3.org/Archives/Public/public-media-
>>>> capture/2012Sep/0003.html [Randell-1]
>>>> http://lists.w3.org/Archives/Public/public-media-
>>>> capture/2012Sep/0001.html [Randell-2]
>>>> http://lists.w3.org/Archives/Public/public-media-
>>>> capture/2012Aug/0044.html [Adam-1]
>>>> http://lists.w3.org/Archives/Public/public-media-
>>>> capture/2012Sep/0004.html [Adam-2]
>>>> http://lists.w3.org/Archives/Public/public-media-
>>>> capture/2012Sep/0006.html [Adam-3]
>>>> http://lists.w3.org/Archives/Public/public-media-
>>>> capture/2012Sep/0005.html [Giridhar-1]
>>>> http://lists.w3.org/Archives/Public/public-media-
>>>> capture/2012Sep/0008.html [Rich-1]
>>>> http://lists.w3.org/Archives/Public/public-media-
>>>> capture/2012Sep/0013.html [Rich-2]
>>>> http://lists.w3.org/Archives/Public/public-media-
>>>> capture/2012Aug/0144.html [Josh-1]
>>>> http://lists.w3.org/Archives/Public/public-media-
>>>> capture/2012Sep/0017.html [Josh-2]
>>>> http://lists.w3.org/Archives/Public/public-media-
>>>> capture/2012Sep/0020.html [Jim-1]
>>>> http://lists.w3.org/Archives/Public/public-media-
>>>> capture/2012Sep/0018.html [Jim-2]
>>>> http://lists.w3.org/Archives/Public/public-media-
>>>> capture/2012Sep/0022.html [Travis-1]
>>>> http://lists.w3.org/Archives/Public/public-media-
>>>> capture/2012Sep/0019.html [Travis-2]
>>>> http://lists.w3.org/Archives/Public/public-media-
>>>> capture/2012Aug/0146.html [Frederick-1]
>>>> http://lists.w3.org/Archives/Public/public-media-
>>>> capture/2012Sep/0021.html
>>>>
>>>> Additionally, I'm thinking about the following: * How often
>>>> will settings be changed in bulk vs. one-at-a-time (with
>> user
>>>> feedback)? * How will developers know to request another device
>>>> via gUM? (how will they know that more devices are available)?
>>>
>>>
>>
>>
>
>

Received on Friday, 5 October 2012 17:27:46 UTC