Re: [mediacapture-main] How to handle changes to the set of tracks in a MediaStream assigned to an element via the srcObject attribute

Good that you bring this up @guidou. When doing some work in this area for Gecko I found the most sensible approach was to:
- for a new track
  - Run through the steps under `If the media resource is found to have an (audio|video) track` in the [resource fetch algorithm](https://www.w3.org/TR/html51/semantics-embedded-content.html#resource-fetch-algorithm)
- for a removed track
  - if the removed track was selected/enabled, for each remaining live track, run through the steps under `If the media resource is found to have an (audio|video) track` in the [resource fetch algorithm](https://www.w3.org/TR/html51/semantics-embedded-content.html#resource-fetch-algorithm)

That said, I am not sure Gecko's implementation fully follows this, though it's probably close. It does help that we don't support AudioTrack and VideoTrack (for now prefed off), or we'd have to deal with issues such as:
- media element playing 1 audio track, 1 video track
- application unselects the video track
- the video track ends (arguably the same issue prevails without this step)
- a new video track is added after a new VideoStreamTrack was added to the MediaStream
- should the new video track be selected (following the steps for creating a VideoTrack) or not (video was intentionally unselected, and it's not clear that this intention has changed)

The only text I find on mutating the TrackLists is on when a track ends, but that doesn't mention whether another track should be enabled or selected. Perhaps I missed something?

@stefhak mentions tracks coming and going when a media element plays a file but to my understanding all tracks of a file src are known when metadata has been loaded, and this is the only case that the spec supports.

I believe that MSE supports tracks that come and go however, so it would be interesting to know how it works there.

-- 
GitHub Notification of comment by Pehrsons
Please view or discuss this issue at https://github.com/w3c/mediacapture-main/issues/453#issuecomment-302127652 using your GitHub account

Received on Wednesday, 17 May 2017 15:30:10 UTC