RE: Early Holiday Present: Settings API version 6

Travis,
  The sequence you describe below (with the muted/unmated events) makes sense.   My question is whether it is something that's _likely_ to happen, or something that the spec should require.  For example:

1. Could there be a camera that could continue streaming while taking a photo?
2.  If there is such a camera, would we want to require that it stop streaming while taking a photo?

In other words, do we want to say that the UA _must_ raise muted/unmated events, or only that it is likely to do so given the way the camera will behave?

- Jim

-----Original Message-----
From: Travis Leithead [mailto:travis.leithead@microsoft.com] 
Sent: Monday, December 17, 2012 1:08 PM
To: Jim Barnett; Adam Bergkvist
Cc: public-media-capture@w3.org
Subject: RE: Early Holiday Present: Settings API version 6

> From: Jim Barnett [mailto:Jim.Barnett@genesyslab.com]
> 
> That's a good question.  If taking a picture involves 'switching the 
> source into "high resolution photo mode"', what happens in other 
> streams that are using the same track?  Do they all suddenly go nuts?  
> This does seem like an operation that must take place on the 
> underlying device, so it can't be isolated to one copy of the Track.

Indeed. This is a source-modification action, so all tracks using this source will be affected. Think about this like how the mirror rotates on an SLR camera--for a brief moment your view must be interrupted in order for the picture to be taken. We can choose to make this as unobservable as possible, or to make it follow to rules that tracks/sources have established. If we assume the latter behavior, then I would expect the following for all tracks affected by the source transition for recording a snapshot:
1. The source temporarily stops -- "muted" events fire on respective tracks because the stream has stopped.
2. The source is reconfigured to match the photo settings (no events fire for this step) 3. The source records/encodes the photo (async -- the "photo" event can fire any time this step asynchronously completes) 4. The source is reconfigured back to the original stored settings (no events fire for this step) 5. The source resumes streaming -- "unmuted" events fire.


Additional responses to Adam's mail below:


> This problem will occur with any operation that changes the settings 
> of the underlying device.  I know we've discussed this, but I forget 
> what the resolution was.
> 
> - Jim
> P.S. The photo-taking functionality is being moved to the recording 
> spec, but the problem is the same.  It's just that now it's my problem 
> rather than yours....
> 
> 
> From: Adam Bergkvist [mailto:adam.bergkvist@ericsson.com]
> 
> On 2012-12-12 20:01, Travis Leithead wrote:
> > http://dvcs.w3.org/hg/dap/raw-file/tip/media-stream-capture/proposal
> > s/
> > SettingsAPI_proposal_v6.html
> >
> 
> I think this look pretty good and I wouldn't object to putting it into 
> the spec.
> 
> Here's some feedback (I haven't had time to read previous feedback so 
> I apologize if I've repeated something already noted.)
> 
>  > Sources that must have an identifier are camera and microphone 
> sources; local file sources are not required to have an identifier. 
> Source identifiers let the application save, identify the availability 
> of, and directly request specific sources.
> 
> In previous discussions we've talked about that applications shouldn't 
> be able to detect if a user has selected a file instead of a "real"
> device. Can we maintain that feature with this proposal?

With source types you already can't know for sure what type the source is. When it's "readonly" that could mean a file or it could me a readonly camera. With unique source id's this anonymity is potentially broken as you note above. How would you propose addressing it? One idea is to give all sources an id, and just ensure that any sources that have id "readonly" get the same id no matter what their back-end [actual] source is. A similar approach could be used with "remote" source types.

On the other hand having these source ids be null offers the same level of anonymity but without any extra work on the user agent.

The other option seems to be to create random identifiers for each readonly/remote source type. That way they aren't distinguishable from actual devices (except that the _real_ devices export their ids from getSourceIds()). 

I don't know how to satisfy Martin's suggestion of unique device ids without making real devices distinguishable from readonly/remote ones.


>  > interface MediaStreamTrack : EventTarget {
>               attribute DOMString           id;
> 
> id should be readonly.

No problem--whatever's in the Media Capture and Stream spec trumps this.


>  > Enumeration description
> new	The track type is new and has not been initialized (connected to a
> source of any kind). This state implies that the track's label will be 
> the empty string.
> 
> s/label/sourceId/ ?

Actually, I intended to say "label". Is label being dropped? Is "kind"? But yes, "sourceId" would also be null in this state.

We do need to say what "label", "id", "enabled" are in the "new" readyState, and whether they change when a source is added/removed.


>  > sourceId of type DOMString, readonly
>      The application-unique identifier for this source. The same 
> identifier must be valid between sessions of this application, but 
> must also be different for other applications. Some sort of GUID is 
> recommended for the identifier.
> 
> s/this source/the source that is providing this track with data/
> 
> It also needs to have a value (null or ""?) when sourceType is "none".

Yes on both points (null rather than "" for consistency with everything else).



>  > interface VideoStreamTrack : MediaStreamTrack {
>      static sequence<DOMString> getSourceIds ();
> 
> I know we've talked about having this functionality on a global object 
> (e.g. navigator.userMedia-something). I'd like to see such a proposal.
> It could also be the place where we attach a listener for new devices 
> (the probe example 7.8 feels a bit hackish).

If the group feels strongly that:
navigator.getUserMediaVideoSourceIds() is better than VideoStreamTrack.getSourceIds() and
navigator.getUserMediaAudioSrouceIds() is better than AudioStreamTrack.getSourceIds() then I'm OK with that. They seem about the same to me from a global object availability perspective.

Also if the use-case for discovering new devices merits it, then two(?) new APIs for requesting when new respective video/audio devices come on line can be easily added (would you want two more(?) APIs for unregistering too?).


>  >  If the sourceType is "photo-camera", then this method temporarily
> (asynchronously) switches the source into "high resolution photo mode"
> 
> What should happen if another track is using this source for video 
> conferencing at the same time?
> 
> 
>  > enum VideoFacingModeEnum {
>      "notavailable",
> 
> Could we use "not-available" instead? (for all occurrences)

No problem here.


>  > interface MediaStreamTrackStateEvent : Event {
>      readonly attribute DOMString[] states;
> 
> Do we need to handle this in such a special way? Couldn't we use a 
> simple event and fire for every state change. That how it normally works.

Yes, we could do that, but the amount of work for this event would then increase linearly with each bit of state that is added. For readyState "none" to anything else, every state will change, and similarly for the "ended" transition. Since these transitions are guaranteed, and other transitions are most often application-initiated (by batches of constraint application), it seemed better to optimize on many states changing at once with minimum event noise. If you do want a minimalist approach then you must decide whether you want the notification to include the state that changed. If not, then a simple "statechanged" event would suffice (with no new Event object definition required). Otherwise, you'll still need the new event object just for "what-changed" state name, and if you want that, then you may as well allow the user agent to batch changes and support the event as defined in V6.


>  > 7.2 Getting access to a specific video source (if available) 
> VideoStreamTrack.getSourceIds().forEach(...
> 
> This example could be simplified a bit with indexOf() to see if the 
> old id exists.

Yeah, Martin suggested "some" (instead of "forEach") which simplifies this nicely.

Received on Monday, 17 December 2012 19:34:09 UTC