Re: [MSE] Rethinking MediaSource.setTrackInfo() and MediaSource.getSourceBuffer()

Hi Aaron, all,

I like this proposal.

Do we already have text about mapping the id attribute of XXXTrack from the media format ? You would need to use this to disambiguate multiple tracks within a multiplexed stream fed into a SourceBuffer.

We could specify the XXXTrack(List) objects we get from the SourceBuffer to be subclasses of the regular ones, with the difference being that we can write the language and kind. That would make it clear that it's only with MediaSource that these things are mutable and they remain immutable on the HTMLMediaElement side. But that might be overzealous.

…Mark



On Jan 10, 2013, at 8:25 AM, Aaron Colwell wrote:

> Hi,
> 
> After spending some time looking at the MSE spec, I'm starting to wonder if we should rethink our decision to create MediaSource.setTrackInfo() and MediaSource.getSourceBuffer(). If I remember correctly, the primary motivation for adding these methods was to provide a way to update the language and kind with data from something like a DASH manifest. I started to think about what the JavaScript would have to look like to accomplish this and it doesn't feel very natural. See below:
> 
> function setAudioTrackInfo(mediaElement, mediaSource, sourceBuffer, language, kind) {
>   for (var i = 0; i <  mediaElement.audioTracks.length; ++i) {
>      var audioTrack = mediaElement.audioTracks[i];
>      if (mediaSource.getSourceBuffer(audioTrack) == sourceBuffer) {
>        mediaSource.setTrackInfo(audioTrack, language, kind);
>      }
>   }
> }
> 
> Something like this feels like it would be much better
> 
> function setAudioTrackInfo(sourceBuffer, language, kind) {
>   for (var i = 0; i <  sourceBuffer.audioTracks.length; ++i) {
>      var audioTrack = sourceBuffer.audioTracks[i];
>      audioTrack.language = language;
>      audioTrack.kind = kind;
>   }
> }
> 
> I know that the other motivation for adding these methods were to avoid having to depend on new features in the HTML spec. I understand and agree with that goal, but I'm starting to feel that the W3C process is shaping the design here more than the actual use case.
> 
> I'd like to propose that we add the following extensions to AudioTrack, VideoTrack, TextTrack, and SourceBuffer in the MSE spec.
> 
> partial interface AudioTrack {
>   attribute DOMString kind;
>   attribute DOMString language;
>   readonly attribute SourceBuffer sourceBuffer;
> }
> 
> partial interface VideoTrack {
>   attribute DOMString kind;
>   attribute DOMString language;
>   readonly attribute SourceBuffer sourceBuffer;
> }
> 
> partial interface TextTrack {
>   attribute DOMString kind;
>   attribute DOMString language;
>   readonly attribute SourceBuffer sourceBuffer;
> }
> 
> partial interface SourceBuffer {
>   readonly attribute AudioTrackList audioTracks;
>   readonly attribute VideoTrackList videoTracks;
>   readonly attribute TextTrackList textTracks;
> }
> 
> I think this will make accessing/modifying this information more natural and maintains the goal of not depending on a specific version of the HTML spec. This simply specifies the requirements for implementations that use MSE. If HTML.next decides to include the new declarations for xxx.kind and xxx.language then that is fine as long as the behavior is consistent between the 2 specs. This shouldn't be a huge problem since only setter behavior is being defined and editors of the 2 specs can coordinate on the definition.
> 
> Obviously I don't want this to derail the FPWD. That ship has sailed, but it may be a good idea to file a bug and update the FPWD candidate with a link to this proposal.
> 
> What do people think about this?
> 
> Aaron

Received on Thursday, 10 January 2013 19:00:35 UTC