RE: How do we enable/select tracks for playout?

>For the audio part it is pretty clear - all tracks that are "enabled" 
>should be used (and mixed) for playout. Video is less so. If there is more than one video track that is "enabled", which one >should be selected? There is no "first".

This was the idea behind adding some way of extending the API to allow setting a primaryVideoTrack.  Setting or changing this value would not produce any difference in behavior in the MediaStream or Track (if the Track is muted, it stays muted), but would serve as an indication to the sink to treat the Track as the 'first'.

- Jim
-----Original Message-----
From: Stefan Håkansson LK [mailto:stefan.lk.hakansson@ericsson.com] 
Sent: Thursday, March 21, 2013 6:24 AM
To: public-media-capture@w3.org
Subject: Re: How do we enable/select tracks for playout?

It seems to me that we are converging on:

* Enable/disable of a MediaStreamTrack is something different from selecting which video track that is being played in a video element
* Those should be separate things (you can select a disabled track for
playout)

This basically means that we don't need to change any of what has already been defined:
* MediaStreamTrack keeps its "enabled" attribute
* media elements allow for selecting a specific video track for playout

The one outstanding question is how to deal with the case of several enabled video tracks in a MediaStream when being attached to a video element. Step 11 of the "resource fetch algorithm" of the media element
(http://www.w3.org/html/wg/drafts/html/master/embedded-content-0.html#concept-media-load-resource)
says:

"If either the media resource or the address of the current media resource indicate a particular set of audio or video tracks to enable, then the selected audio tracks must be enabled in the element's audioTracks object, and, of the selected video tracks, the one that is listed first in the element's videoTracks object must be selected."

For the audio part it is pretty clear - all tracks that are "enabled" 
should be used (and mixed) for playout. Video is less so. If there is more than one video track that is "enabled", which one should be selected? There is no "first".

Stefan


On 3/21/13 7:46 AM, Adam Bergkvist wrote:
> On 2013-03-20 18:12, Martin Thomson wrote:
>> On 20 March 2013 07:33, Adam Bergkvist <adam.bergkvist@ericsson.com>
>> wrote:
>>> On 2013-03-19 14:00, Jim Barnett wrote:
>>>>
>>>> The final decision of what to display should be up to the sink.  If 
>>>> there are sinks that can handle MediaStreams containing multiple 
>>>> audio or video tracks, then they should be able to use whatever 
>>>> heuristics they want to select the Track(s) to play out.
>>
>> I'm concerned with the non-deterministic behaviour that could happen 
>> if we don't make something explicit here.
>
> I agree. We need to do something about this since we removed the order 
> of the tracks and there's no logical "first" track of each kind anymore.
>
> But if the sink has some mechanism for enumerating tracks and 
> selecting which one to render, then that should just work for 
> MediaStreams as well. However, selecting a track is no guarantee that 
> there's data on it; it might just be muted on track-level anyhow.
>
> /Adam
>

Received on Thursday, 21 March 2013 12:30:29 UTC