Re: How do we enable/select tracks for playout?

On 2013-03-19 12:55, Harald Alvestrand wrote:
> On 03/19/2013 12:39 PM, Stefan Håkansson LK wrote:
>> I'd like to discuss enable/disable of MediaStreamTrack's a bit, and
>> specifically how it works with the media element.
>>
>> Currently, the app can enable/disable a MediaStreamTrack by toggling
>> the "enabled" attribute [1].
>>
>> If we look at the video case, you get the impression that you could
>> create a MediaStream with two video tracks (e.g. front and rear
>> camera) and then toggle which one of them that is displayed in a video
>> element by enabling/disabling the different tracks.
>>
>> But that is not the case. The current definition of videoTrackList and
>> videoTrack for the media element in the html draft [2] instead says
>> that you'll need to set the "selected" attribute of the videoTrack you
>> want to play in the videoTrackList of the _video element_. Toggling
>> the enabled bit of the MediaStreamTrack's will not change which track
>> that is played (if I understand correctly).
>
> I think you're right - toggling the enabled bit should play out black /
> silence, not change which one is played out.
>
>>
>> So, in a scenario where the video is sent over the network (using
>> PeerConnection), and the sending app want to change which video is
>> played at the remote end, it would actually have to signal to the app
>> at the remote end to set "selected" on the other videoTrack on the
>> video element. Toggling enabled/disabled at the sending side will not
>> help.
>>
>> I've previously argued that this is strange, and that we should remove
>> enable/disable from MediaStreamTrack (and instead have it on consumers
>> - Recorder, PeerConnection, media element - only). This could be one
>> way to go.
>
> Doing that at the track level would require that you have an interface
> to access the tracks inside the streams connected to the consumer for
> every possible type of consumer. That seems strange to me.

We already have it for the media element and for the Recorder. And I 
suspect we will need an API to access the tracks inside a MediaStream 
for the PeerConnection as well - to set things like priority, bitrate 
limits and other things related to the transport. But I am not sure that 
is the place we should do enable/disable.

>
>>
>> Another could be that we keep enable/disable on the MediaStreamTrack,
>> but instead redefine how media elements work when the source is a
>> MediaStream. It could be defined so that the enabled MediaStreamTrack
>> is always played, and if more than one track is enabled one of the
>> enabled ones is (randomly) picked for playout.
>
> That seems appealing to me - since we abandoned the track index, we
> can't specify that the first will play out, since there is no "first".

I also like this one - with the caveat that we will need reliable and 
fast signaling of enable/disable. It should not be that browser A says 
"200 ms and no RTP - this is probably disabled" while browser B uses 
400ms. Another problem is the random behavior - hard to create tests if 
nothing else.

>
>>
>> A third could be that we keep the current way. I find it a bit
>> confusing, but it could work. Note (again in the network case) that if
>> we have reliable signaling/detection of disabled/muted, the receiving
>> application could listen to "onmute", and if that fires at the
>> currently playing video MediaStreamTrack the app could select another
>> for play in the video elements videoTrackList.
>>
>> What do you think?
>
> I'd like to have more opinions here...

+1

>
>>
>> Stefan
>>
>> [1] http://dev.w3.org/2011/webrtc/editor/getusermedia.html#attributes-1
>> [2]
>> http://www.w3.org/html/wg/drafts/html/master/embedded-content-0.html#audiotracklist-and-videotracklist-objects
>>
>>
>
>

Received on Tuesday, 19 March 2013 12:56:02 UTC