Re: Updated Media Capture and Streams editor's draft

(Cherry-picking some questions to comment on)

On 5/3/13 1:36 PM, Mandyam, Giridhar wrote:
> Thanks; the current draft looks really good.
>
> Comments/questions:
>
> Terminology:
> a) Source:  "Sources do not have constraints ...".  Does this imply anything about whether sources have settings, and whether those settings can be changed by the end user?

I like to say that users have constraints. Users specify their 
constraints, and in doing so, indirectly affect the source's settings 
(which are a shared resource).

> b) State (Source State):  "A source's state must always conform to the current set of mandatory constraints ...".  Why is this requirement necessary?  If the implementation can comply with a set of mandatory constraints that are not in conformance with the source's state, then this should be acceptable behavior.  For instance, if a source is limited to 10 fps but an implementation can satisfy 30 fps through some form of e.g. motion interpolation - then why should this be prohibited by the spec?

I seem to recall something about a UA being allowed to produce more 
modes out of a mode, provided it didn't make up bits. E.g. removing 
every other frame would be OK, but interpolating new ones would not be.

Regardless, doesn't "source" here include the UA (and it's faked modes)?

> c) Constraints: " It is possible for two tracks that share a unique source to apply contradictory constraints. Under such contradictions, the implementation will mute both tracks and notify them that they are over-constrained. " This needs to be re-worded.  Contradictory optional constraints should not lead to an over-constrained situation.

I would say that optional constraints arguably don't contradict each 
other (notwithstanding the oxymoron they are).

> Section 4
>
> a) Section 4.1:  " Both MediaStream and MediaStreamTrack objects can be cloned. This allows for greater control since the separate instances can be manipulated and consumed individually. "  Can we be more specific and call these structured clones, as defined in http://www.w3.org/html/wg/drafts/html/master/infrastructure.html#safe-passing-of-structured-data?  Any attempt to actually manipulate mandatory constraints on a cloned track would seem to result in an overconstrained situation, so the idea of a track clone being anything other than a structured clone seems not to be possible.  I would appreciate an explanation if I am incorrect.

I think you are incorrect. :-) Constraints can be bounds and ranges, 
which act as fences for allowed values of settings. Constraints can also 
be ideal values which act as attractors. So as I understand it, you can 
move your fences and attractors, and I can move mine (in another tab 
competing for the same camera at the same time). As long as there is 
overlap that allows the UA to keep satisfying both of us, then no 
overconstrained event should take place, and settings may even change 
(constraints can be loosened as well, not just tightened). Replace "me" 
with a track you cloned, and the same should hold true. i.e. I assume a 
clone to be a first-class citizen.

> b) Section 4.1:  " When a MediaStream object is being generated from a local file ...".  If we are discussing more futuristic features, why not also include generation of a MediaStream from a <video>/<audio> tag as well?

+1 and lets stop talking about "a local file".

> c) Section 4.2:  I think the MediaStream attribute should be called 'finished' instead of 'ended', if anything so as to distinguish the final state of a MediaStream object from the final state of its constituent tracks.  But this may have already been discussed within the group; if so, I don't want to re-open discussion on this topic because it is trivial.
>
> d) Section 4.2:  Should the MediaStream TrackID be immutable?  The text is silent on this, but there seems to be a potential issue using with the MediaStream constructor if the track ID's as a result of MediaStream creation.  For instance, if the MediaStreamTrackSequence has to video tracks uniquely identified by ID, then the developer may have an expectation that those ID's are preserved when a MediaStream is created from this track sequence.   I looked through the recent cloning discussion on the mail list but I couldn't find whether this was ever addressed.
>
> e) Section 4.2.2: Similar question as previous comment for when clone() is invoked.  The stream ID is different for the clone, but are TrackID's preserved in the clone?

No, it's a deep clone apparently with new IDs for everybody (see clone's 
processing model in 4.2.3).

> f) Section 4.2: I don't understand who the consumer is for the MediaStream ID.  This ID can be created and be globally unique, but I am not understanding why it needs to be exposed as an attribute on the stream object.  This may have been discussed before in the group - if so, I don't want to unnecessarily re-open a resolved issue.
>
> g) Section 4.2.2: We could collapse getVideoTracks() and getAudioTracks() into a single method, e.g. getTracks (optional DOMString type), where type can be "video" or "audio".
>
> h) Section 4.3.1:  For completeness, it would be good to include a mention on memory management (i.e. when can a  stream or track can be GC'ed or deleted, e.g. see https://dvcs.w3.org/hg/audio/raw-file/tip/webaudio/specification.html#lifetime-AudioNode)
>
> i) Section 4.3.3.1:  id attribute.  Taken with http://tools.ietf.org/html/draft-alvestrand-mmusic-msid-02, it would seem that the W3C has a different interpretation of MediaStreamTrack ID from the IETF.  Should the msid and MediaStreamTrack ID's be equivalent?
>
> j) Section 4.3.4:  I had commented separately that it would be good to get rid of the "photo-camera" source type and instead raise an error when takePhoto() is called on a source incapable of taking photos.

No comment.

> Section 11
>
> a) I'd remove Example 2.  We have takePhoto() and getFrame() methods defined now, so there is no need to do things this way.  I think an example about applying constraints to a MediaStreamTrack after it has been created might be a more useful example.

+1

> Section 12
>
> a) I thought "zoom" was considered an acceptable track constraint.  It is important for camera preview streams.  Can it be added back in?  I realize that a preview stream can be "zoomed" using CSS transforms (e.g. scale:  ), but this may not be an accurate imitation of the camera zoom operation.
>
> b) VideoFacingModeEnum was not defined in the doc anywhere (at least I could not find it).

http://w3c.github.io/mediacapture-main/getusermedia.html#idl-def-VideoFacingModeEnum 
?

How could you miss that wonderful drawing?

.: Jan-Ivar :.

> -----Original Message-----
> From: Dan Burnett [mailto:dburnett@voxeo.com]
> Sent: Monday, April 29, 2013 10:07 PM
> To: public-media-capture@w3.org
> Subject: Updated Media Capture and Streams editor's draft
>
> Hi
>
> A new version of the editor's draft is available.
>
> Dated version: http://dev.w3.org/2011/webrtc/editor/archives/20130429/getusermedia.html
> Living document: http://dev.w3.org/2011/webrtc/editor/getusermedia.html  (yes, this shows a date of 20130430 -- this is an artifact of publishing after midnight Boston time, so ignore it)
>
>
> Changes include:
> 	* Added readonly and remote attributes to MediaStreamTrack
> 	* Removed getConstraint(), setConstraint(), appendConstraint(), and prependConstraint().
> 	* Added source states. Added states() method on tracks. Moved sourceType and sourceId to be states.
> 	* Added source capabilities. Added capabilities() method on tracks.
> 	* Added clarifying text about MediaStreamTrack lifecycle and mediaflow.
> 	* Made MediaStreamTrack cloning explicit.
> 	* Removed takePhoto() and friends from VideoStreamTrack (we have a separate Image Capture Spec).
> 	* Made getUserMedia() error callback mandatory.
>
> Please review and provide feedback.
>
> Dan Burnett (for the editors)

Received on Wednesday, 6 May 2015 01:40:58 UTC