RE: How do we enable/select tracks for playout?

The final decision of what to display should be up to the sink.  If there are sinks that can handle MediaStreams containing multiple audio or video tracks, then they should be able to use whatever heuristics they want to select the Track(s) to play out. 

It is useful to have a mute/disable function on the sending side, the substitutes black frames for video, just like 'mute' in audio.  This shouldn't necessarily have any impact on the sink (though if it's smart enough to detect the muting and wants to switch to another track, that's fine).

The big question is what to do when we have a MediaStream that contains multiple Tracks, and a sink that can play out only one. In that case, we could say that the real interface to the sink is the Track, and leave it up to the API to select the Track that it wants to pass to the sink.  Alternatively, if we there are reasons to keep MediaStream as the interface, we could add the concept of 'defaultVideoTrack' and 'defaultAudioTrack' to the MediaStream.  This wouldn't change the behavior the sending end at all, but simply be a signal "If you are streamed into a sink that can only handle a single Track, use this one".

- Jim

-----Original Message-----
From: Harald Alvestrand [mailto:harald@alvestrand.no] 
Sent: Tuesday, March 19, 2013 7:56 AM
To: public-media-capture@w3.org
Subject: Re: How do we enable/select tracks for playout?

On 03/19/2013 12:39 PM, Stefan Håkansson LK wrote:
> I'd like to discuss enable/disable of MediaStreamTrack's a bit, and 
> specifically how it works with the media element.
>
> Currently, the app can enable/disable a MediaStreamTrack by toggling 
> the "enabled" attribute [1].
>
> If we look at the video case, you get the impression that you could 
> create a MediaStream with two video tracks (e.g. front and rear
> camera) and then toggle which one of them that is displayed in a video 
> element by enabling/disabling the different tracks.
>
> But that is not the case. The current definition of videoTrackList and 
> videoTrack for the media element in the html draft [2] instead says 
> that you'll need to set the "selected" attribute of the videoTrack you 
> want to play in the videoTrackList of the _video element_. Toggling 
> the enabled bit of the MediaStreamTrack's will not change which track 
> that is played (if I understand correctly).

I think you're right - toggling the enabled bit should play out black / silence, not change which one is played out.

>
> So, in a scenario where the video is sent over the network (using 
> PeerConnection), and the sending app want to change which video is 
> played at the remote end, it would actually have to signal to the app 
> at the remote end to set "selected" on the other videoTrack on the 
> video element. Toggling enabled/disabled at the sending side will not 
> help.
>
> I've previously argued that this is strange, and that we should remove 
> enable/disable from MediaStreamTrack (and instead have it on consumers
> - Recorder, PeerConnection, media element - only). This could be one 
> way to go.

Doing that at the track level would require that you have an interface to access the tracks inside the streams connected to the consumer for every possible type of consumer. That seems strange to me.

>
> Another could be that we keep enable/disable on the MediaStreamTrack, 
> but instead redefine how media elements work when the source is a 
> MediaStream. It could be defined so that the enabled MediaStreamTrack 
> is always played, and if more than one track is enabled one of the 
> enabled ones is (randomly) picked for playout.

That seems appealing to me - since we abandoned the track index, we 
can't specify that the first will play out, since there is no "first".

>
> A third could be that we keep the current way. I find it a bit 
> confusing, but it could work. Note (again in the network case) that if 
> we have reliable signaling/detection of disabled/muted, the receiving 
> application could listen to "onmute", and if that fires at the 
> currently playing video MediaStreamTrack the app could select another 
> for play in the video elements videoTrackList.
>
> What do you think?

I'd like to have more opinions here...

>
> Stefan
>
> [1] http://dev.w3.org/2011/webrtc/editor/getusermedia.html#attributes-1
> [2] 
> http://www.w3.org/html/wg/drafts/html/master/embedded-content-0.html#audiotracklist-and-videotracklist-objects
>

Received on Tuesday, 19 March 2013 13:01:25 UTC