Re: Updated Media Capture and Streams editor's draft

Some more comments.....

On 05/06/2015 03:40 AM, Jan-Ivar Bruaroey wrote:
> (Cherry-picking some questions to comment on)
>
> On 5/3/13 1:36 PM, Mandyam, Giridhar wrote:
>> Thanks; the current draft looks really good.
>>
>> Comments/questions:
>>
>> Terminology:
>> a) Source:  "Sources do not have constraints ...".  Does this imply
>> anything about whether sources have settings, and whether those
>> settings can be changed by the end user?
>
> I like to say that users have constraints. Users specify their
> constraints, and in doing so, indirectly affect the source's settings
> (which are a shared resource).
>
>> b) State (Source State):  "A source's state must always conform to
>> the current set of mandatory constraints ...".  Why is this
>> requirement necessary?  If the implementation can comply with a set
>> of mandatory constraints that are not in conformance with the
>> source's state, then this should be acceptable behavior.  For
>> instance, if a source is limited to 10 fps but an implementation can
>> satisfy 30 fps through some form of e.g. motion interpolation - then
>> why should this be prohibited by the spec?
>
> I seem to recall something about a UA being allowed to produce more
> modes out of a mode, provided it didn't make up bits. E.g. removing
> every other frame would be OK, but interpolating new ones would not be.
>
> Regardless, doesn't "source" here include the UA (and it's faked modes)?

Yes, my interpretation is that the "source" is "what the system does".

And motion interpolation comes under "making up bits" in my book.

>
>> c) Constraints: " It is possible for two tracks that share a unique
>> source to apply contradictory constraints. Under such contradictions,
>> the implementation will mute both tracks and notify them that they
>> are over-constrained. " This needs to be re-worded.  Contradictory
>> optional constraints should not lead to an over-constrained situation.
>
> I would say that optional constraints arguably don't contradict each
> other (notwithstanding the oxymoron they are).

The name is somewhat self-contradictory; I've interpreted it as "a
request that happens to use the same linguistic form as a constraint".

Note: We don't have a construct called "optional" any more. There are
hard constraints (exact and min/max), indications of desire (ideal), and
"constraints that it's possible to skip" (advanced).

>
>> Section 4
>>
>> a) Section 4.1:  " Both MediaStream and MediaStreamTrack objects can
>> be cloned. This allows for greater control since the separate
>> instances can be manipulated and consumed individually. "  Can we be
>> more specific and call these structured clones, as defined in
>> http://www.w3.org/html/wg/drafts/html/master/infrastructure.html#safe-passing-of-structured-data? 
>> Any attempt to actually manipulate mandatory constraints on a cloned
>> track would seem to result in an overconstrained situation, so the
>> idea of a track clone being anything other than a structured clone
>> seems not to be possible.  I would appreciate an explanation if I am
>> incorrect.
>
> I think you are incorrect. :-) Constraints can be bounds and ranges,
> which act as fences for allowed values of settings. Constraints can
> also be ideal values which act as attractors. So as I understand it,
> you can move your fences and attractors, and I can move mine (in
> another tab competing for the same camera at the same time). As long
> as there is overlap that allows the UA to keep satisfying both of us,
> then no overconstrained event should take place, and settings may even
> change (constraints can be loosened as well, not just tightened).
> Replace "me" with a track you cloned, and the same should hold true.
> i.e. I assume a clone to be a first-class citizen.

This is already specified in the introduction to section 4:

" A cloned ||MediaStreamTrack|
<http://w3c.github.io/mediacapture-main/getusermedia.html#idl-def-MediaStreamTrack>| has
a set of constraints
<http://w3c.github.io/mediacapture-main/getusermedia.html#constrainable-interface> that
is independent of the instance it is cloned from, which allows media
from the same source to have different constraints applied for
different consumer
<http://w3c.github.io/mediacapture-main/getusermedia.html#dfn-consumer>s."

It's also specified in the description of MediaStreamTrack.Clone:

http://w3c.github.io/mediacapture-main/getusermedia.html#dom-mediastreamtrack-clone

Is any clarification needed?

>
>> b) Section 4.1:  " When a MediaStream object is being generated from
>> a local file ...".  If we are discussing more futuristic features,
>> why not also include generation of a MediaStream from a
>> <video>/<audio> tag as well?
>
> +1 and lets stop talking about "a local file".

Generation of a MediaStream from a <video>/<audio> tag is documented as
an extension, specified in this document:

http://w3c.github.io/mediacapture-fromelement/

Any remaining reference to "a local file" needs to be flagged as a bug;
we've agreed to remove this idea entirely, since generation from a media
element fulfils the need, and saves us from getting into a whole thicket
of possible new needed specifications.
>
>> c) Section 4.2:  I think the MediaStream attribute should be called
>> 'finished' instead of 'ended', if anything so as to distinguish the
>> final state of a MediaStream object from the final state of its
>> constituent tracks.  But this may have already been discussed within
>> the group; if so, I don't want to re-open discussion on this topic
>> because it is trivial.
>>
>> d) Section 4.2:  Should the MediaStream TrackID be immutable?  The
>> text is silent on this, but there seems to be a potential issue using
>> with the MediaStream constructor if the track ID's as a result of
>> MediaStream creation.  For instance, if the MediaStreamTrackSequence
>> has to video tracks uniquely identified by ID, then the developer may
>> have an expectation that those ID's are preserved when a MediaStream
>> is created from this track sequence.   I looked through the recent
>> cloning discussion on the mail list but I couldn't find whether this
>> was ever addressed.
>>
>> e) Section 4.2.2: Similar question as previous comment for when
>> clone() is invoked.  The stream ID is different for the clone, but
>> are TrackID's preserved in the clone?
>
> No, it's a deep clone apparently with new IDs for everybody (see
> clone's processing model in 4.2.3).
>
>> f) Section 4.2: I don't understand who the consumer is for the
>> MediaStream ID.  This ID can be created and be globally unique, but I
>> am not understanding why it needs to be exposed as an attribute on
>> the stream object.  This may have been discussed before in the group
>> - if so, I don't want to unnecessarily re-open a resolved issue.
>>
>> g) Section 4.2.2: We could collapse getVideoTracks() and
>> getAudioTracks() into a single method, e.g. getTracks (optional
>> DOMString type), where type can be "video" or "audio".
>>
>> h) Section 4.3.1:  For completeness, it would be good to include a
>> mention on memory management (i.e. when can a  stream or track can be
>> GC'ed or deleted, e.g. see
>> https://dvcs.w3.org/hg/audio/raw-file/tip/webaudio/specification.html#lifetime-AudioNode)
>>
>> i) Section 4.3.3.1:  id attribute.  Taken with
>> http://tools.ietf.org/html/draft-alvestrand-mmusic-msid-02, it would
>> seem that the W3C has a different interpretation of MediaStreamTrack
>> ID from the IETF.  Should the msid and MediaStreamTrack ID's be
>> equivalent?
>>
>> j) Section 4.3.4:  I had commented separately that it would be good
>> to get rid of the "photo-camera" source type and instead raise an
>> error when takePhoto() is called on a source incapable of taking photos.
>
> No comment.
>
>> Section 11
>>
>> a) I'd remove Example 2.  We have takePhoto() and getFrame() methods
>> defined now, so there is no need to do things this way.  I think an
>> example about applying constraints to a MediaStreamTrack after it has
>> been created might be a more useful example.
>
> +1
>
>> Section 12
>>
>> a) I thought "zoom" was considered an acceptable track constraint. 
>> It is important for camera preview streams.  Can it be added back
>> in?  I realize that a preview stream can be "zoomed" using CSS
>> transforms (e.g. scale:  ), but this may not be an accurate imitation
>> of the camera zoom operation.

Zoom is a complex operation. In the interest of keeping the
specification simple for version 1.0, it was removed at our Boston meeting.

>>
>> b) VideoFacingModeEnum was not defined in the doc anywhere (at least
>> I could not find it).
>
> http://w3c.github.io/mediacapture-main/getusermedia.html#idl-def-VideoFacingModeEnum
> ?
>
> How could you miss that wonderful drawing?
>
> .: Jan-Ivar :.
>
>> -----Original Message-----
>> From: Dan Burnett [mailto:dburnett@voxeo.com]
>> Sent: Monday, April 29, 2013 10:07 PM
>> To: public-media-capture@w3.org
>> Subject: Updated Media Capture and Streams editor's draft
>>
>> Hi
>>
>> A new version of the editor's draft is available.
>>
>> Dated version:
>> http://dev.w3.org/2011/webrtc/editor/archives/20130429/getusermedia.html
>> Living document:
>> http://dev.w3.org/2011/webrtc/editor/getusermedia.html  (yes, this
>> shows a date of 20130430 -- this is an artifact of publishing after
>> midnight Boston time, so ignore it)
>>
>>
>> Changes include:
>>     * Added readonly and remote attributes to MediaStreamTrack
>>     * Removed getConstraint(), setConstraint(), appendConstraint(),
>> and prependConstraint().
>>     * Added source states. Added states() method on tracks. Moved
>> sourceType and sourceId to be states.
>>     * Added source capabilities. Added capabilities() method on tracks.
>>     * Added clarifying text about MediaStreamTrack lifecycle and
>> mediaflow.
>>     * Made MediaStreamTrack cloning explicit.
>>     * Removed takePhoto() and friends from VideoStreamTrack (we have
>> a separate Image Capture Spec).
>>     * Made getUserMedia() error callback mandatory.
>>
>> Please review and provide feedback.
>>
>> Dan Burnett (for the editors)
>


-- 
Surveillance is pervasive. Go Dark.