Re: How do we enable/select tracks for playout?

On 3/21/13 3:05 PM, Jim Barnett wrote:
> Wait a minute... what if primaryVideoTrack was only valid locally?
> (In other words, it wouldn't be signaled to the other side over a
> PeerConnection.)  It would still be useful to let the application
> define the Track to display, rather than letting the sink guess.  In
> this case, changing the value of primaryVideoTrack would cause the
> sink to change which track it was displaying.  So Peer1 sends
> multiple video streams to Peer2, and the code on Peer2's side uses
> primaryVideoTrack to toggle which Track its local video element
> displays. Peer1 wouldn't know about this, but that should be ok.

I am not convinced really. If it is only local, why can't the app select 
at the sink directly (e.g. by using the selected attribute on the 
VideoTrackList on the media element)?

I think the value would be for the remote case.

>
> - Jim
>
> -----Original Message----- From: Jim Barnett
> [mailto:Jim.Barnett@genesyslab.com] Sent: Thursday, March 21, 2013
> 9:47 AM To: Stefan Håkansson LK Cc: public-media-capture@w3.org
> Subject: RE: How do we enable/select tracks for playout?
>
> Stefan, It would certainly be simpler - for implementers - to leave
> the selection of the video track up to the sink or to let it be
> random.  This would lead to indeterminacy in applications, and to
> possible non-portability of apps (in the sense that a given app might
> work well with Firefox's random choice, but not Chrome's).
>
> One possibility would be to leave it out for now, and add it later if
> non-portability turns out to be a practical, and not just
> hypothetical, problem.
>
> - Jim
>
> -----Original Message----- From: Stefan Håkansson LK
> [mailto:stefan.lk.hakansson@ericsson.com] Sent: Thursday, March 21,
> 2013 9:39 AM To: Jim Barnett Cc: public-media-capture@w3.org Subject:
> Re: How do we enable/select tracks for playout?
>
> On 3/21/13 1:30 PM, Jim Barnett wrote:
>>> For the audio part it is pretty clear - all tracks that are
>>> "enabled" should be used (and mixed) for playout. Video is less
>>> so. If there is more than one video track that is "enabled",
>>> which one
>>>> should be selected? There is no "first".
>>
>> This was the idea behind adding some way of extending the API to
>> allow setting a primaryVideoTrack.  Setting or changing this value
>> would not produce any difference in behavior in the MediaStream or
>> Track (if the Track is muted, it stays muted), but would serve as
>> an indication to the sink to treat the Track as the 'first'.
>
> What would the consequences be if we did not add this, but instead
> let the track to select be random?
>
> I'm asking only because I can see two issues with a
> "primaryVideoTrack":
>
> 1. In the networked case, it is one more thing we need to convey to
> the other side. Knowing all the problems we have with the signaling I
> would like to avoid adding things...
>
> 2. If we have a "primaryVideoTrack" setting that can be associated
> with one video track in a MediaStream, then it is not far fetched
> that this setting can be changed (to indicate another track as
> "primary") in operation - and then you could in principle have one
> track being "primary" in the MediaStream and another being "selected"
> in the sink (media element). I think that would be confusing.
>
> Br, Stefan
>
>
>>
>> - Jim -----Original Message----- From: Stefan Håkansson LK
>> [mailto:stefan.lk.hakansson@ericsson.com] Sent: Thursday, March
>> 21, 2013 6:24 AM To: public-media-capture@w3.org Subject: Re: How
>> do we enable/select tracks for playout?
>>
>> It seems to me that we are converging on:
>>
>> * Enable/disable of a MediaStreamTrack is something different from
>> selecting which video track that is being played in a video element
>> * Those should be separate things (you can select a disabled track
>> for playout)
>>
>> This basically means that we don't need to change any of what has
>> already been defined: * MediaStreamTrack keeps its "enabled"
>> attribute * media elements allow for selecting a specific video
>> track for playout
>>
>> The one outstanding question is how to deal with the case of
>> several enabled video tracks in a MediaStream when being attached
>> to a video element. Step 11 of the "resource fetch algorithm" of
>> the media element
>> (http://www.w3.org/html/wg/drafts/html/master/embedded-content-0.html#
>>
>>
concept-media-load-resource)
>>
>>
> says:
>>
>> "If either the media resource or the address of the current media
>> resource indicate a particular set of audio or video tracks to
>> enable, then the selected audio tracks must be enabled in the
>> element's audioTracks object, and, of the selected video tracks,
>> the one that is listed first in the element's videoTracks object
>> must be selected."
>>
>> For the audio part it is pretty clear - all tracks that are
>> "enabled" should be used (and mixed) for playout. Video is less
>> so. If there is more than one video track that is "enabled", which
>> one should be selected? There is no "first".
>>
>> Stefan
>>
>>
>> On 3/21/13 7:46 AM, Adam Bergkvist wrote:
>>> On 2013-03-20 18:12, Martin Thomson wrote:
>>>> On 20 March 2013 07:33, Adam Bergkvist
>>>> <adam.bergkvist@ericsson.com> wrote:
>>>>> On 2013-03-19 14:00, Jim Barnett wrote:
>>>>>>
>>>>>> The final decision of what to display should be up to the
>>>>>> sink. If there are sinks that can handle MediaStreams
>>>>>> containing multiple audio or video tracks, then they should
>>>>>> be able to use whatever heuristics they want to select the
>>>>>> Track(s) to play out.
>>>>
>>>> I'm concerned with the non-deterministic behaviour that could
>>>> happen if we don't make something explicit here.
>>>
>>> I agree. We need to do something about this since we removed the
>>> order of the tracks and there's no logical "first" track of each
>>> kind anymore.
>>>
>>> But if the sink has some mechanism for enumerating tracks and
>>> selecting which one to render, then that should just work for
>>> MediaStreams as well. However, selecting a track is no guarantee
>>> that there's data on it; it might just be muted on track-level
>>> anyhow.
>>>
>>> /Adam
>>>
>>
>>
>

Received on Thursday, 21 March 2013 14:10:24 UTC