Re: [Bug 22785] there should be a way to schedule audio/video track selection changes in HTMLMediaElements

(bringing this to the attention of public-html at the suggestion of the Media Source Extensions chair)

As part of the Media Source Extensions (MSE) work, we identified a potential general enhancement that would have applicability beyond MSE, and that therefore shouldn't be addressed in the MSE spec or an MSE-specific way.

One use case for MSE is insertion of an advertisement into a media stream. It will often be the case that an advertisement will have a different track configuration from the main program. For example: a main program may comprise demultiplexed video and audio stream files, represented in MSE as two SourceBuffers, with one SourceBuffer contributing a VideoTrack and one contributing an AudioTrack. However, the advertisement may comprise a single multiplexed video and audio stream file, represented in MSE as one SourceBuffer contributing both a VideoTrack and an AudioTrack.

Currently, selecting and deselecting the video and audio tracks of an HTMLMediaElement from JavaScript takes effect immediately.  However, JavaScript execution can't be guaranteed to be synchronized to within one video frame or one audio sample, for a seamless transition from the tracks of a main program to the tracks of an advertisement. JavaScript execution latency could cause media playback stalls or glitches.

A general solution put forward by an editor of the MSE spec is to introduce a means to schedule the selection/deslection of video and audio tracks on the media playback timeline, so that the media engine itself can effect seamless track transitions without a time-critical JavaScript execution.

I opened bug 22785 to track this proposed solution, in response to MSE bug 22137.

-michael thornburgh

Received on Tuesday, 23 July 2013 22:39:20 UTC