Re: How do we enable/select tracks for playout?

On 3/21/13 1:30 PM, Jim Barnett wrote:
>> For the audio part it is pretty clear - all tracks that are
>> "enabled" should be used (and mixed) for playout. Video is less so.
>> If there is more than one video track that is "enabled", which one
>> >should be selected? There is no "first".
>
> This was the idea behind adding some way of extending the API to
> allow setting a primaryVideoTrack.  Setting or changing this value
> would not produce any difference in behavior in the MediaStream or
> Track (if the Track is muted, it stays muted), but would serve as an
> indication to the sink to treat the Track as the 'first'.

What would the consequences be if we did not add this, but instead let 
the track to select be random?

I'm asking only because I can see two issues with a "primaryVideoTrack":

1. In the networked case, it is one more thing we need to convey to the 
other side. Knowing all the problems we have with the signaling I would 
like to avoid adding things...

2. If we have a "primaryVideoTrack" setting that can be associated with 
one video track in a MediaStream, then it is not far fetched that this 
setting can be changed (to indicate another track as "primary") in 
operation - and then you could in principle have one track being 
"primary" in the MediaStream and another being "selected" in the sink 
(media element). I think that would be confusing.

Br,
Stefan


>
> - Jim -----Original Message----- From: Stefan Håkansson LK
> [mailto:stefan.lk.hakansson@ericsson.com] Sent: Thursday, March 21,
> 2013 6:24 AM To: public-media-capture@w3.org Subject: Re: How do we
> enable/select tracks for playout?
>
> It seems to me that we are converging on:
>
> * Enable/disable of a MediaStreamTrack is something different from
> selecting which video track that is being played in a video element *
> Those should be separate things (you can select a disabled track for
> playout)
>
> This basically means that we don't need to change any of what has
> already been defined: * MediaStreamTrack keeps its "enabled"
> attribute * media elements allow for selecting a specific video track
> for playout
>
> The one outstanding question is how to deal with the case of several
> enabled video tracks in a MediaStream when being attached to a video
> element. Step 11 of the "resource fetch algorithm" of the media
> element
> (http://www.w3.org/html/wg/drafts/html/master/embedded-content-0.html#concept-media-load-resource)
>
>
says:
>
> "If either the media resource or the address of the current media
> resource indicate a particular set of audio or video tracks to
> enable, then the selected audio tracks must be enabled in the
> element's audioTracks object, and, of the selected video tracks, the
> one that is listed first in the element's videoTracks object must be
> selected."
>
> For the audio part it is pretty clear - all tracks that are
> "enabled" should be used (and mixed) for playout. Video is less so.
> If there is more than one video track that is "enabled", which one
> should be selected? There is no "first".
>
> Stefan
>
>
> On 3/21/13 7:46 AM, Adam Bergkvist wrote:
>> On 2013-03-20 18:12, Martin Thomson wrote:
>>> On 20 March 2013 07:33, Adam Bergkvist
>>> <adam.bergkvist@ericsson.com> wrote:
>>>> On 2013-03-19 14:00, Jim Barnett wrote:
>>>>>
>>>>> The final decision of what to display should be up to the
>>>>> sink.  If there are sinks that can handle MediaStreams
>>>>> containing multiple audio or video tracks, then they should
>>>>> be able to use whatever heuristics they want to select the
>>>>> Track(s) to play out.
>>>
>>> I'm concerned with the non-deterministic behaviour that could
>>> happen if we don't make something explicit here.
>>
>> I agree. We need to do something about this since we removed the
>> order of the tracks and there's no logical "first" track of each
>> kind anymore.
>>
>> But if the sink has some mechanism for enumerating tracks and
>> selecting which one to render, then that should just work for
>> MediaStreams as well. However, selecting a track is no guarantee
>> that there's data on it; it might just be muted on track-level
>> anyhow.
>>
>> /Adam
>>
>
>

Received on Thursday, 21 March 2013 13:39:14 UTC