Which videoTrack should be played in the <Video> element?

The spec currently does not specify which videoTrack should be displayed in the <Video> element when the video element is consuming a MediaStream. When there is only one MediaStreamTrack in the videoTrack list, this is obvious. But it is somewhat less obvious what should happen when there are either:

1)      No videoTracks (e.g., the user requested or received a MediaStream with only audioTracks present)

2)      More than one videoTracks that are enabled (enabled-by-default as the spec says)

May I suggest that the spec needs to address these two cases:

1.       How should the video element consumer handle audio-only MediaStreams?

2.       How should multiple videoTrack's be addressed when consumed by a video element?

For the multiple-video tracks case, the HTML5 video element handles this by toggling the tracks in its videoTracks list (http://dev.w3.org/html5/spec/single-page.html#videotracklist) such that only one video track is active at a time; it has the concept of a "selected" track and a "selectedIndex".

My proposal for MediaStreams is that audio-only MediaStreams (being consumed by a Video element) should result in a zero-sized media dimension, and we should be sure to note in the spec that the media element's videoWidth/videoHeight attributes are set to zero. For MediaStreams with multiple videoTracks, I suggest that we either adopt to model used by HTML5, or make the order of the Tracks significant, such that when there are multiple enabled/active video tracks in a MediaStream, the first active/enabled video track is the one that is played in the video element. Users can then set the enabled attribute to false, so that the video element falls back to the next available video track. For this scenario it might also be nice to furnish a "setEnabled" or "mute" API on the MediaStreamTrackList that takes a single unsigned long index as a parameter and makes the track at that given index enabled while disabling all other tracks.

Received on Friday, 27 July 2012 21:48:10 UTC