RE: How do we enable/select tracks for playout?

Sounds good to me.

- Jim

-----Original Message-----
From: Stefan Håkansson LK [mailto:stefan.lk.hakansson@ericsson.com] 
Sent: Thursday, March 21, 2013 10:23 AM
To: public-media-capture@w3.org
Subject: Re: How do we enable/select tracks for playout?

On 3/21/13 3:19 PM, Harald Alvestrand wrote:
> On 03/21/2013 02:46 PM, Jim Barnett wrote:
>> Stefan,
>>    It would certainly be simpler - for implementers - to leave the 
>> selection of the video track up to the sink or to let it be random.
>> This would lead to indeterminacy in applications, and to possible 
>> non-portability of apps (in the sense that a given app might work 
>> well with Firefox's random choice, but not Chrome's).
>>
>> One possibility would be to leave it out for now, and add it later if 
>> non-portability turns out to be a practical, and not just 
>> hypothetical, problem.
>
> Another possibility is to make it deterministic but non-useful; for 
> instance, we might specify that if the sequence of tracks is needed, 
> one sorts the tracks on ID (guaranteed to be distinct and well 
> specified, and all browsers will behave the same for the same IDs, but 
> the result is random enough that applications will take steps to pick 
> the Right Track).

I was having similar thoughts.

>
>>
>> - Jim
>>
>> -----Original Message-----
>> From: Stefan Håkansson LK [mailto:stefan.lk.hakansson@ericsson.com]
>> Sent: Thursday, March 21, 2013 9:39 AM
>> To: Jim Barnett
>> Cc: public-media-capture@w3.org
>> Subject: Re: How do we enable/select tracks for playout?
>>
>> On 3/21/13 1:30 PM, Jim Barnett wrote:
>>>> For the audio part it is pretty clear - all tracks that are "enabled"
>>>> should be used (and mixed) for playout. Video is less so.
>>>> If there is more than one video track that is "enabled", which one
>>>>> should be selected? There is no "first".
>>> This was the idea behind adding some way of extending the API to 
>>> allow setting a primaryVideoTrack.  Setting or changing this value 
>>> would not produce any difference in behavior in the MediaStream or 
>>> Track (if the Track is muted, it stays muted), but would serve as an 
>>> indication to the sink to treat the Track as the 'first'.
>> What would the consequences be if we did not add this, but instead 
>> let the track to select be random?
>>
>> I'm asking only because I can see two issues with a "primaryVideoTrack":
>>
>> 1. In the networked case, it is one more thing we need to convey to 
>> the other side. Knowing all the problems we have with the signaling I 
>> would like to avoid adding things...
>>
>> 2. If we have a "primaryVideoTrack" setting that can be associated 
>> with one video track in a MediaStream, then it is not far fetched 
>> that this setting can be changed (to indicate another track as 
>> "primary") in operation - and then you could in principle have one 
>> track being "primary" in the MediaStream and another being "selected" 
>> in the sink (media element). I think that would be confusing.
>>
>> Br,
>> Stefan
>>
>>
>>> - Jim -----Original Message----- From: Stefan Håkansson LK 
>>> [mailto:stefan.lk.hakansson@ericsson.com] Sent: Thursday, March 21,
>>> 2013 6:24 AM To: public-media-capture@w3.org Subject: Re: How do we 
>>> enable/select tracks for playout?
>>>
>>> It seems to me that we are converging on:
>>>
>>> * Enable/disable of a MediaStreamTrack is something different from 
>>> selecting which video track that is being played in a video element 
>>> * Those should be separate things (you can select a disabled track 
>>> for
>>> playout)
>>>
>>> This basically means that we don't need to change any of what has 
>>> already been defined: * MediaStreamTrack keeps its "enabled"
>>> attribute * media elements allow for selecting a specific video 
>>> track for playout
>>>
>>> The one outstanding question is how to deal with the case of several 
>>> enabled video tracks in a MediaStream when being attached to a video 
>>> element. Step 11 of the "resource fetch algorithm" of the media 
>>> element 
>>> (http://www.w3.org/html/wg/drafts/html/master/embedded-content-0.htm

>>> l#
>>> concept-media-load-resource)
>>>
>>>
>> says:
>>> "If either the media resource or the address of the current media 
>>> resource indicate a particular set of audio or video tracks to 
>>> enable, then the selected audio tracks must be enabled in the 
>>> element's audioTracks object, and, of the selected video tracks, the 
>>> one that is listed first in the element's videoTracks object must be selected."
>>>
>>> For the audio part it is pretty clear - all tracks that are "enabled"
>>> should be used (and mixed) for playout. Video is less so.
>>> If there is more than one video track that is "enabled", which one 
>>> should be selected? There is no "first".
>>>
>>> Stefan
>>>
>>>
>>> On 3/21/13 7:46 AM, Adam Bergkvist wrote:
>>>> On 2013-03-20 18:12, Martin Thomson wrote:
>>>>> On 20 March 2013 07:33, Adam Bergkvist 
>>>>> <adam.bergkvist@ericsson.com>
>>>>> wrote:
>>>>>> On 2013-03-19 14:00, Jim Barnett wrote:
>>>>>>> The final decision of what to display should be up to the sink.
>>>>>>> If there are sinks that can handle MediaStreams containing 
>>>>>>> multiple audio or video tracks, then they should be able to use 
>>>>>>> whatever heuristics they want to select the
>>>>>>> Track(s) to play out.
>>>>> I'm concerned with the non-deterministic behaviour that could 
>>>>> happen if we don't make something explicit here.
>>>> I agree. We need to do something about this since we removed the 
>>>> order of the tracks and there's no logical "first" track of each 
>>>> kind anymore.
>>>>
>>>> But if the sink has some mechanism for enumerating tracks and 
>>>> selecting which one to render, then that should just work for 
>>>> MediaStreams as well. However, selecting a track is no guarantee 
>>>> that there's data on it; it might just be muted on track-level anyhow.
>>>>
>>>> /Adam
>>>>
>>>
>
>

Received on Thursday, 21 March 2013 14:25:34 UTC