W3C home > Mailing lists > Public > public-media-capture@w3.org > December 2012

Re: Early Holiday Present: Settings API version 6

From: Martin Thomson <martin.thomson@gmail.com>
Date: Tue, 18 Dec 2012 11:12:12 -0800
Message-ID: <CABkgnnUH3KCa-pjSW4SkszXA9rtmt77Y5GF1iWnx_Vo6o9SK0A@mail.gmail.com>
To: Adam Bergkvist <adam.bergkvist@ericsson.com>
Cc: Travis Leithead <travis.leithead@microsoft.com>, Jim Barnett <Jim.Barnett@genesyslab.com>, "public-media-capture@w3.org" <public-media-capture@w3.org>
I'm inclined to address this differently.  Either:

1. Provide a "file" device placeholder on all browsers (crappy user
experience because now we get false positives when apps check to see
if the user has devices)
2. Create a new device ID for files when the user opts to create the
file, which to the app appears to be a new device that was "plugged
in" in response to their request for a mic/camera.

I prefer the second, which is very much like your fourth, with one
difference: getDeviceIds() still works.  Apps can't enumerate files (a
good thing), but that's OK.  There are several up-sides over what you
suggest):
 - apps can't request a specific file
 - apps can still see what "real" devices are present prior to
requesting permission (necessary for some use cases)
 - users can choose a file if the app permits it, at which point we
can decide how much to share with the app about that file

I'm still in two minds as to whether there is one device ID for files
altogether or whether there is a different device ID for each file.
I'm leaning toward the former.

I still believe that hiding device IDs until after consent is granted
is overly cautious.  It prevents some application usage scenarios.

On 18 December 2012 00:41, Adam Bergkvist <adam.bergkvist@ericsson.com> wrote:
> Hi
>
> Answers in line.
>
>
> On 2012-12-17 19:08, Travis Leithead wrote:
>>>
>>> From: Jim Barnett [mailto:Jim.Barnett@genesyslab.com]
>>>
>>> That's a good question.  If taking a picture involves 'switching the
>>> source into "high resolution photo mode"', what happens in other streams
>>> that are using the same track?  Do they all suddenly go nuts?  This does
>>> seem like an operation that must take place on the underlying device, so
>>> it can't be isolated to one copy of the Track.
>>
>>
>> Indeed. This is a source-modification action, so all tracks using this
>> source will be affected. Think about this like how the mirror rotates on an
>> SLR camera--for a brief moment your view must be interrupted in order for
>> the picture to be taken. We can choose to make this as unobservable as
>> possible, or to make it follow to rules that tracks/sources have
>> established. If we assume the latter behavior, then I would expect the
>> following for all tracks affected by the source transition for recording a
>> snapshot:
>> 1. The source temporarily stops -- "muted" events fire on respective
>> tracks because the stream has stopped.
>> 2. The source is reconfigured to match the photo settings (no events fire
>> for this step)
>> 3. The source records/encodes the photo (async -- the "photo" event can
>> fire any time this step asynchronously completes)
>> 4. The source is reconfigured back to the original stored settings (no
>> events fire for this step)
>> 5. The source resumes streaming -- "unmuted" events fire.
>>
>>
>> Additional responses to Adam's mail below:
>>
>>
>>> This problem will occur with any operation that changes the settings of
>>> the underlying device.  I know we've discussed this, but I forget what
>>> the
>>> resolution was.
>>>
>>> - Jim
>>> P.S. The photo-taking functionality is being moved to the recording spec,
>>> but the problem is the same.  It's just that now it's my problem rather
>>> than yours....
>>>
>>>
>>> From: Adam Bergkvist [mailto:adam.bergkvist@ericsson.com]
>>>
>>> On 2012-12-12 20:01, Travis Leithead wrote:
>>>>
>>>> http://dvcs.w3.org/hg/dap/raw-file/tip/media-stream-capture/proposals/
>>>> SettingsAPI_proposal_v6.html
>>>>
>>>
>>> I think this look pretty good and I wouldn't object to putting it into
>>> the
>>> spec.
>>>
>>> Here's some feedback (I haven't had time to read previous feedback so I
>>> apologize if I've repeated something already noted.)
>>>
>>>   > Sources that must have an identifier are camera and microphone
>>> sources;
>>> local file sources are not required to have an identifier. Source
>>> identifiers let the application save, identify the availability of, and
>>> directly request specific sources.
>>>
>>> In previous discussions we've talked about that applications shouldn't be
>>> able to detect if a user has selected a file instead of a "real"
>>> device. Can we maintain that feature with this proposal?
>>
>>
>> With source types you already can't know for sure what type the
>> source is. When it's "readonly" that could mean a file or it could me
>> a readonly camera. With unique source id's this anonymity is
>> potentially broken as you note above. How would you propose
>> addressing it? One idea is to give all sources an id, and just ensure
>> that any sources that have id "readonly" get the same id no matter
>> what their back-end [actual] source is. A similar approach could be
>> used with "remote" source types.
>>
>> On the other hand having these source ids be null offers the same
>> level of anonymity but without any extra work on the user agent.
>>
>> The other option seems to be to create random identifiers for each
>> readonly/remote source type. That way they aren't distinguishable
>> from actual devices (except that the _real_ devices export their ids
>> from getSourceIds()).
>
>
> No matter how we solve the id issue, a source type of "readonly" will always
> tell the app that it's either a file or a readonly camera. Perhaps we can't
> get around that. Even if we didn't expose the information explicitly, the
> app could probe this information by trying to apply a set of constraints to
> change the source, observe the result and detect the "readonly" behavior.
>
> Summary of id proposals from your text above:
> 1) Same ids for all readonly sources
> 2) Null ids...
> 3) Unique ids but not present in getSourceIds() list
>
> I see an issue with 1) and 2) in the following scenario. An app uses a
> device of type "camera" and stores its deviceId for future use (the original
> use-case for device id). The user uses the same app the next day, however
> the requested camera is now "owned" by an other application and is therefore
> returned to this app as a "readonly" source. If we want to support this, the
> source needs to maintain its unique id even though it's readonly and the id
> should also be present in the list returned from getSourceIds().
>
> A fourth alternative would be to give individual files a unique id as well
> which would make them *almost* indistinguishable from "real" readonly
> devices. I say "almost" since a real device could at some later session
> suddenly have the type "camera" if it became available. (We could avoid this
> by saying that a device can never change from "readonly" to something else.)
> The problem with with an approach like this (with unique ids for files) is
> that it's incompatible with getDeviceIds(). Obviously we can't pre-generate
> a deviceId for every possible file and expose that in the list. I suggested
> an approach earlier where we don't expose getDeviceIds(), but instead let
> the app "build" its own deviceId list with devices it has been granted
> permission to use previously. Unique ids for both real devices and files
> would then be generated and exposed when permissions have been granted with
> getUserMedia() (and not before). If we remove getDeviceIds() we would,
> however, need another method of saying that the UA has audio and video
> devices. If this satisfies our use cases it would also be a bit more
> defensive from an information leakage point of view. (It doesn't help much
> wrt fingerprinting though as Martin stated in a different mail).
>
>
>> I don't know how to satisfy Martin's suggestion of unique device ids
>> without making real devices distinguishable from readonly/remote
>> ones.
>>
>>
>>>   > interface MediaStreamTrack : EventTarget {
>>>                attribute DOMString           id;
>>>
>>> id should be readonly.
>>
>>
>> No problem--whatever's in the Media Capture and Stream spec trumps this.
>>
>>
>>>   > Enumeration description
>>> new     The track type is new and has not been initialized (connected to
>>> a
>>> source of any kind). This state implies that the track's label will be
>>> the empty string.
>>>
>>> s/label/sourceId/ ?
>>
>>
>> Actually, I intended to say "label". Is label being dropped? Is
>> "kind"? But yes, "sourceId" would also be null in this state.
>
>
> You're right, this applies to label as well since it's describing the source
> in a human readable form. Nope, label for tracks is still there. We're
> currently discussing if we should have a label of the same kind for streams
> as well (and what that should be in that case).
>
>
>> We do need to say what "label", "id", "enabled" are in the "new"
>> readyState, and whether they change when a source is added/removed.
>>
>>
>>>   > sourceId of type DOMString, readonly
>>>       The application-unique identifier for this source. The same
>>> identifier must be valid between sessions of this application, but must
>>> also be different for other applications. Some sort of GUID is
>>> recommended for the identifier.
>>>
>>> s/this source/the source that is providing this track with data/
>>>
>>> It also needs to have a value (null or ""?) when sourceType is "none".
>>
>>
>> Yes on both points (null rather than "" for consistency with everything
>> else).
>>
>>
>>
>>>   > interface VideoStreamTrack : MediaStreamTrack {
>>>       static sequence<DOMString> getSourceIds ();
>>>
>>> I know we've talked about having this functionality on a global object
>>> (e.g. navigator.userMedia-something). I'd like to see such a proposal.
>>> It could also be the place where we attach a listener for new devices
>>> (the probe example 7.8 feels a bit hackish).
>>
>>
>> If the group feels strongly that:
>> navigator.getUserMediaVideoSourceIds() is better than
>> VideoStreamTrack.getSourceIds()
>> and
>> navigator.getUserMediaAudioSrouceIds() is better than
>> AudioStreamTrack.getSourceIds()
>> then I'm OK with that. They seem about the same to me from a global object
>> availability perspective.
>>
>> Also if the use-case for discovering new devices merits it, then two(?)
>> new APIs for requesting when new respective video/audio devices come on line
>> can be easily added (would you want two more(?) APIs for unregistering
>> too?).
>>
>>
>>>   >  If the sourceType is "photo-camera", then this method temporarily
>>> (asynchronously) switches the source into "high resolution photo mode"
>>>
>>> What should happen if another track is using this source for video
>>> conferencing at the same time?
>>>
>>>
>>>   > enum VideoFacingModeEnum {
>>>       "notavailable",
>>>
>>> Could we use "not-available" instead? (for all occurrences)
>>
>>
>> No problem here.
>>
>>
>>>   > interface MediaStreamTrackStateEvent : Event {
>>>       readonly attribute DOMString[] states;
>>>
>>> Do we need to handle this in such a special way? Couldn't we use a
>>> simple event and fire for every state change. That how it normally works.
>>
>>
>> Yes, we could do that, but the amount of work for this event would
>> then increase linearly with each bit of state that is added. For
>> readyState "none" to anything else, every state will change, and
>> similarly for the "ended" transition. Since these transitions are
>> guaranteed, and other transitions are most often
>> application-initiated (by batches of constraint application), it
>> seemed better to optimize on many states changing at once with
>> minimum event noise. If you do want a minimalist approach then you
>> must decide whether you want the notification to include the state
>> that changed. If not, then a simple "statechanged" event would
>> suffice (with no new Event object definition required). Otherwise,
>> you'll still need the new event object just for "what-changed" state
>> name, and if you want that, then you may as well allow the user agent
>> to batch changes and support the event as defined in V6.
>
>
> I'm not sure I fully understand this. It feels like sourceType going from
> "none" to something else and other things happening to the source will just
> affect the existing readyState of the track. I don't think we want to expose
> several states in parallel that monitors the data flow in the track.
>
> I can see a case when constraints are changed and we want to fire an event
> for this. In that case the track was "live" and after the change it's (if
> successful) still "live". But couldn't we have a separate event for that
> that's not related to the "life cycle" of the track?
>
> /Adam
>
>
>>
>>>   > 7.2 Getting access to a specific video source (if available)
>>> VideoStreamTrack.getSourceIds().forEach(...
>>>
>>> This example could be simplified a bit with indexOf() to see if the old
>>> id exists.
>>
>>
>> Yeah, Martin suggested "some" (instead of "forEach") which simplifies this
>> nicely.
>>
>
>
Received on Tuesday, 18 December 2012 19:12:41 UTC

This archive was generated by hypermail 2.4.0 : Friday, 17 January 2020 16:26:13 UTC