W3C home > Mailing lists > Public > public-media-capture@w3.org > March 2013

Re: How do we enable/select tracks for playout?

From: Stefan Håkansson LK <stefan.lk.hakansson@ericsson.com>
Date: Thu, 21 Mar 2013 15:23:18 +0100
Message-ID: <514B17D6.10108@ericsson.com>
To: public-media-capture@w3.org
On 3/21/13 3:19 PM, Harald Alvestrand wrote:
> On 03/21/2013 02:46 PM, Jim Barnett wrote:
>> Stefan,
>>    It would certainly be simpler - for implementers - to leave the
>> selection of the video track up to the sink or to let it be random.
>> This would lead to indeterminacy in applications, and to possible
>> non-portability of apps (in the sense that a given app might work well
>> with Firefox's random choice, but not Chrome's).
>>
>> One possibility would be to leave it out for now, and add it later if
>> non-portability turns out to be a practical, and not just
>> hypothetical, problem.
>
> Another possibility is to make it deterministic but non-useful; for
> instance, we might specify that if the sequence of tracks is needed, one
> sorts the tracks on ID (guaranteed to be distinct and well specified,
> and all browsers will behave the same for the same IDs, but the result
> is random enough that applications will take steps to pick the Right
> Track).

I was having similar thoughts.

>
>>
>> - Jim
>>
>> -----Original Message-----
>> From: Stefan Håkansson LK [mailto:stefan.lk.hakansson@ericsson.com]
>> Sent: Thursday, March 21, 2013 9:39 AM
>> To: Jim Barnett
>> Cc: public-media-capture@w3.org
>> Subject: Re: How do we enable/select tracks for playout?
>>
>> On 3/21/13 1:30 PM, Jim Barnett wrote:
>>>> For the audio part it is pretty clear - all tracks that are "enabled"
>>>> should be used (and mixed) for playout. Video is less so.
>>>> If there is more than one video track that is "enabled", which one
>>>>> should be selected? There is no "first".
>>> This was the idea behind adding some way of extending the API to allow
>>> setting a primaryVideoTrack.  Setting or changing this value would not
>>> produce any difference in behavior in the MediaStream or Track (if the
>>> Track is muted, it stays muted), but would serve as an indication to
>>> the sink to treat the Track as the 'first'.
>> What would the consequences be if we did not add this, but instead let
>> the track to select be random?
>>
>> I'm asking only because I can see two issues with a "primaryVideoTrack":
>>
>> 1. In the networked case, it is one more thing we need to convey to
>> the other side. Knowing all the problems we have with the signaling I
>> would like to avoid adding things...
>>
>> 2. If we have a "primaryVideoTrack" setting that can be associated
>> with one video track in a MediaStream, then it is not far fetched that
>> this setting can be changed (to indicate another track as "primary")
>> in operation - and then you could in principle have one track being
>> "primary" in the MediaStream and another being "selected" in the sink
>> (media element). I think that would be confusing.
>>
>> Br,
>> Stefan
>>
>>
>>> - Jim -----Original Message----- From: Stefan Håkansson LK
>>> [mailto:stefan.lk.hakansson@ericsson.com] Sent: Thursday, March 21,
>>> 2013 6:24 AM To: public-media-capture@w3.org Subject: Re: How do we
>>> enable/select tracks for playout?
>>>
>>> It seems to me that we are converging on:
>>>
>>> * Enable/disable of a MediaStreamTrack is something different from
>>> selecting which video track that is being played in a video element *
>>> Those should be separate things (you can select a disabled track for
>>> playout)
>>>
>>> This basically means that we don't need to change any of what has
>>> already been defined: * MediaStreamTrack keeps its "enabled"
>>> attribute * media elements allow for selecting a specific video track
>>> for playout
>>>
>>> The one outstanding question is how to deal with the case of several
>>> enabled video tracks in a MediaStream when being attached to a video
>>> element. Step 11 of the "resource fetch algorithm" of the media
>>> element
>>> (http://www.w3.org/html/wg/drafts/html/master/embedded-content-0.html#
>>> concept-media-load-resource)
>>>
>>>
>> says:
>>> "If either the media resource or the address of the current media
>>> resource indicate a particular set of audio or video tracks to enable,
>>> then the selected audio tracks must be enabled in the element's
>>> audioTracks object, and, of the selected video tracks, the one that is
>>> listed first in the element's videoTracks object must be selected."
>>>
>>> For the audio part it is pretty clear - all tracks that are "enabled"
>>> should be used (and mixed) for playout. Video is less so.
>>> If there is more than one video track that is "enabled", which one
>>> should be selected? There is no "first".
>>>
>>> Stefan
>>>
>>>
>>> On 3/21/13 7:46 AM, Adam Bergkvist wrote:
>>>> On 2013-03-20 18:12, Martin Thomson wrote:
>>>>> On 20 March 2013 07:33, Adam Bergkvist <adam.bergkvist@ericsson.com>
>>>>> wrote:
>>>>>> On 2013-03-19 14:00, Jim Barnett wrote:
>>>>>>> The final decision of what to display should be up to the sink.
>>>>>>> If there are sinks that can handle MediaStreams containing
>>>>>>> multiple audio or video tracks, then they should be able to use
>>>>>>> whatever heuristics they want to select the
>>>>>>> Track(s) to play out.
>>>>> I'm concerned with the non-deterministic behaviour that could happen
>>>>> if we don't make something explicit here.
>>>> I agree. We need to do something about this since we removed the
>>>> order of the tracks and there's no logical "first" track of each kind
>>>> anymore.
>>>>
>>>> But if the sink has some mechanism for enumerating tracks and
>>>> selecting which one to render, then that should just work for
>>>> MediaStreams as well. However, selecting a track is no guarantee that
>>>> there's data on it; it might just be muted on track-level anyhow.
>>>>
>>>> /Adam
>>>>
>>>
>
>
Received on Thursday, 21 March 2013 14:23:47 UTC

This archive was generated by hypermail 2.4.0 : Friday, 17 January 2020 16:26:15 UTC