Re: Early Holiday Present: Settings API version 6

Hi

Answers in line.

On 2012-12-17 19:08, Travis Leithead wrote:
>> From: Jim Barnett [mailto:Jim.Barnett@genesyslab.com]
>>
>> That's a good question.  If taking a picture involves 'switching the
>> source into "high resolution photo mode"', what happens in other streams
>> that are using the same track?  Do they all suddenly go nuts?  This does
>> seem like an operation that must take place on the underlying device, so
>> it can't be isolated to one copy of the Track.
>
> Indeed. This is a source-modification action, so all tracks using this source will be affected. Think about this like how the mirror rotates on an SLR camera--for a brief moment your view must be interrupted in order for the picture to be taken. We can choose to make this as unobservable as possible, or to make it follow to rules that tracks/sources have established. If we assume the latter behavior, then I would expect the following for all tracks affected by the source transition for recording a snapshot:
> 1. The source temporarily stops -- "muted" events fire on respective tracks because the stream has stopped.
> 2. The source is reconfigured to match the photo settings (no events fire for this step)
> 3. The source records/encodes the photo (async -- the "photo" event can fire any time this step asynchronously completes)
> 4. The source is reconfigured back to the original stored settings (no events fire for this step)
> 5. The source resumes streaming -- "unmuted" events fire.
>
>
> Additional responses to Adam's mail below:
>
>
>> This problem will occur with any operation that changes the settings of
>> the underlying device.  I know we've discussed this, but I forget what the
>> resolution was.
>>
>> - Jim
>> P.S. The photo-taking functionality is being moved to the recording spec,
>> but the problem is the same.  It's just that now it's my problem rather
>> than yours....
>>
>>
>> From: Adam Bergkvist [mailto:adam.bergkvist@ericsson.com]
>>
>> On 2012-12-12 20:01, Travis Leithead wrote:
>>> http://dvcs.w3.org/hg/dap/raw-file/tip/media-stream-capture/proposals/
>>> SettingsAPI_proposal_v6.html
>>>
>>
>> I think this look pretty good and I wouldn't object to putting it into the
>> spec.
>>
>> Here's some feedback (I haven't had time to read previous feedback so I
>> apologize if I've repeated something already noted.)
>>
>>   > Sources that must have an identifier are camera and microphone sources;
>> local file sources are not required to have an identifier. Source
>> identifiers let the application save, identify the availability of, and
>> directly request specific sources.
>>
>> In previous discussions we've talked about that applications shouldn't be
>> able to detect if a user has selected a file instead of a "real"
>> device. Can we maintain that feature with this proposal?
>
> With source types you already can't know for sure what type the
> source is. When it's "readonly" that could mean a file or it could me
> a readonly camera. With unique source id's this anonymity is
> potentially broken as you note above. How would you propose
> addressing it? One idea is to give all sources an id, and just ensure
> that any sources that have id "readonly" get the same id no matter
> what their back-end [actual] source is. A similar approach could be
> used with "remote" source types.
>
> On the other hand having these source ids be null offers the same
> level of anonymity but without any extra work on the user agent.
>
> The other option seems to be to create random identifiers for each
> readonly/remote source type. That way they aren't distinguishable
> from actual devices (except that the _real_ devices export their ids
> from getSourceIds()).

No matter how we solve the id issue, a source type of "readonly" will 
always tell the app that it's either a file or a readonly camera. 
Perhaps we can't get around that. Even if we didn't expose the 
information explicitly, the app could probe this information by trying 
to apply a set of constraints to change the source, observe the result 
and detect the "readonly" behavior.

Summary of id proposals from your text above:
1) Same ids for all readonly sources
2) Null ids...
3) Unique ids but not present in getSourceIds() list

I see an issue with 1) and 2) in the following scenario. An app uses a 
device of type "camera" and stores its deviceId for future use (the 
original use-case for device id). The user uses the same app the next 
day, however the requested camera is now "owned" by an other application 
and is therefore returned to this app as a "readonly" source. If we want 
to support this, the source needs to maintain its unique id even though 
it's readonly and the id should also be present in the list returned 
from getSourceIds().

A fourth alternative would be to give individual files a unique id as 
well which would make them *almost* indistinguishable from "real" 
readonly devices. I say "almost" since a real device could at some later 
session suddenly have the type "camera" if it became available. (We 
could avoid this by saying that a device can never change from 
"readonly" to something else.) The problem with with an approach like 
this (with unique ids for files) is that it's incompatible with 
getDeviceIds(). Obviously we can't pre-generate a deviceId for every 
possible file and expose that in the list. I suggested an approach 
earlier where we don't expose getDeviceIds(), but instead let the app 
"build" its own deviceId list with devices it has been granted 
permission to use previously. Unique ids for both real devices and files 
would then be generated and exposed when permissions have been granted 
with getUserMedia() (and not before). If we remove getDeviceIds() we 
would, however, need another method of saying that the UA has audio and 
video devices. If this satisfies our use cases it would also be a bit 
more defensive from an information leakage point of view. (It doesn't 
help much wrt fingerprinting though as Martin stated in a different mail).

> I don't know how to satisfy Martin's suggestion of unique device ids
> without making real devices distinguishable from readonly/remote
> ones.
>
>
>>   > interface MediaStreamTrack : EventTarget {
>>                attribute DOMString           id;
>>
>> id should be readonly.
>
> No problem--whatever's in the Media Capture and Stream spec trumps this.
>
>
>>   > Enumeration description
>> new	The track type is new and has not been initialized (connected to a
>> source of any kind). This state implies that the track's label will be
>> the empty string.
>>
>> s/label/sourceId/ ?
>
> Actually, I intended to say "label". Is label being dropped? Is
> "kind"? But yes, "sourceId" would also be null in this state.

You're right, this applies to label as well since it's describing the 
source in a human readable form. Nope, label for tracks is still there. 
We're currently discussing if we should have a label of the same kind 
for streams as well (and what that should be in that case).

> We do need to say what "label", "id", "enabled" are in the "new"
> readyState, and whether they change when a source is added/removed.
>
>
>>   > sourceId of type DOMString, readonly
>>       The application-unique identifier for this source. The same
>> identifier must be valid between sessions of this application, but must
>> also be different for other applications. Some sort of GUID is
>> recommended for the identifier.
>>
>> s/this source/the source that is providing this track with data/
>>
>> It also needs to have a value (null or ""?) when sourceType is "none".
>
> Yes on both points (null rather than "" for consistency with everything else).
>
>
>
>>   > interface VideoStreamTrack : MediaStreamTrack {
>>       static sequence<DOMString> getSourceIds ();
>>
>> I know we've talked about having this functionality on a global object
>> (e.g. navigator.userMedia-something). I'd like to see such a proposal.
>> It could also be the place where we attach a listener for new devices
>> (the probe example 7.8 feels a bit hackish).
>
> If the group feels strongly that:
> navigator.getUserMediaVideoSourceIds() is better than VideoStreamTrack.getSourceIds()
> and
> navigator.getUserMediaAudioSrouceIds() is better than AudioStreamTrack.getSourceIds()
> then I'm OK with that. They seem about the same to me from a global object availability perspective.
>
> Also if the use-case for discovering new devices merits it, then two(?) new APIs for requesting when new respective video/audio devices come on line can be easily added (would you want two more(?) APIs for unregistering too?).
>
>
>>   >  If the sourceType is "photo-camera", then this method temporarily
>> (asynchronously) switches the source into "high resolution photo mode"
>>
>> What should happen if another track is using this source for video
>> conferencing at the same time?
>>
>>
>>   > enum VideoFacingModeEnum {
>>       "notavailable",
>>
>> Could we use "not-available" instead? (for all occurrences)
>
> No problem here.
>
>
>>   > interface MediaStreamTrackStateEvent : Event {
>>       readonly attribute DOMString[] states;
>>
>> Do we need to handle this in such a special way? Couldn't we use a
>> simple event and fire for every state change. That how it normally works.
>
> Yes, we could do that, but the amount of work for this event would
> then increase linearly with each bit of state that is added. For
> readyState "none" to anything else, every state will change, and
> similarly for the "ended" transition. Since these transitions are
> guaranteed, and other transitions are most often
> application-initiated (by batches of constraint application), it
> seemed better to optimize on many states changing at once with
> minimum event noise. If you do want a minimalist approach then you
> must decide whether you want the notification to include the state
> that changed. If not, then a simple "statechanged" event would
> suffice (with no new Event object definition required). Otherwise,
> you'll still need the new event object just for "what-changed" state
> name, and if you want that, then you may as well allow the user agent
> to batch changes and support the event as defined in V6.

I'm not sure I fully understand this. It feels like sourceType going 
from "none" to something else and other things happening to the source 
will just affect the existing readyState of the track. I don't think we 
want to expose several states in parallel that monitors the data flow in 
the track.

I can see a case when constraints are changed and we want to fire an 
event for this. In that case the track was "live" and after the change 
it's (if successful) still "live". But couldn't we have a separate event 
for that that's not related to the "life cycle" of the track?

/Adam

>
>>   > 7.2 Getting access to a specific video source (if available)
>> VideoStreamTrack.getSourceIds().forEach(...
>>
>> This example could be simplified a bit with indexOf() to see if the old
>> id exists.
>
> Yeah, Martin suggested "some" (instead of "forEach") which simplifies this nicely.
>

Received on Tuesday, 18 December 2012 08:41:46 UTC