Re: [MSE] Rethinking MediaSource.setTrackInfo() and MediaSource.getSourceBuffer()

Comments inline


On Thu, Jan 10, 2013 at 11:00 AM, Mark Watson <watsonm@netflix.com> wrote:

> Hi Aaron, all,
>
> I like this proposal.
>
> Do we already have text about mapping the id attribute of XXXTrack from
> the media format ? You would need to use this to disambiguate multiple
> tracks within a multiplexed stream fed into a SourceBuffer.
>

If this proposal was accepted, I was planning on submitting a follow-on
proposal of a containerid property that contains the trackID used in the
container. I didn't include it here because I wanted to treat that as a
seperate issue and I just wanted to address the functionality covered by
these 2 methods. I figured once the precedent was set for overriding the
existing definitions, then it would be easy to add this extra property.


>
> We could specify the XXXTrack(List) objects we get from the SourceBuffer
> to be subclasses of the regular ones, with the difference being that we can
> write the language and kind. That would make it clear that it's only with
> MediaSource that these things are mutable and they remain immutable on the
> HTMLMediaElement side. But that might be overzealous.
>

I thought about that, but then you have WritableAudioTrack or some such
thing and then you need a WriteAudioTrackList...yada..yada..yada. It
doesn't seem to be worth adding the extra classes when we essentially want
the existing classes, just slightly modified in a way that can be
considered backwards compatible w/ the HTML5 definitions. Hopefully the new
definitions will make their way into HTML.next or what ever it gets called,
but I think this is a safe way to proceed w/o getting blocked on the HTML
spec's progress.

Aaron


>
> …Mark
>
>
>
> On Jan 10, 2013, at 8:25 AM, Aaron Colwell wrote:
>
> > Hi,
> >
> > After spending some time looking at the MSE spec, I'm starting to wonder
> if we should rethink our decision to create MediaSource.setTrackInfo() and
> MediaSource.getSourceBuffer(). If I remember correctly, the primary
> motivation for adding these methods was to provide a way to update the
> language and kind with data from something like a DASH manifest. I started
> to think about what the JavaScript would have to look like to accomplish
> this and it doesn't feel very natural. See below:
> >
> > function setAudioTrackInfo(mediaElement, mediaSource, sourceBuffer,
> language, kind) {
> >   for (var i = 0; i <  mediaElement.audioTracks.length; ++i) {
> >      var audioTrack = mediaElement.audioTracks[i];
> >      if (mediaSource.getSourceBuffer(audioTrack) == sourceBuffer) {
> >        mediaSource.setTrackInfo(audioTrack, language, kind);
> >      }
> >   }
> > }
> >
> > Something like this feels like it would be much better
> >
> > function setAudioTrackInfo(sourceBuffer, language, kind) {
> >   for (var i = 0; i <  sourceBuffer.audioTracks.length; ++i) {
> >      var audioTrack = sourceBuffer.audioTracks[i];
> >      audioTrack.language = language;
> >      audioTrack.kind = kind;
> >   }
> > }
> >
> > I know that the other motivation for adding these methods were to avoid
> having to depend on new features in the HTML spec. I understand and agree
> with that goal, but I'm starting to feel that the W3C process is shaping
> the design here more than the actual use case.
> >
> > I'd like to propose that we add the following extensions to AudioTrack,
> VideoTrack, TextTrack, and SourceBuffer in the MSE spec.
> >
> > partial interface AudioTrack {
> >   attribute DOMString kind;
> >   attribute DOMString language;
> >   readonly attribute SourceBuffer sourceBuffer;
> > }
> >
> > partial interface VideoTrack {
> >   attribute DOMString kind;
> >   attribute DOMString language;
> >   readonly attribute SourceBuffer sourceBuffer;
> > }
> >
> > partial interface TextTrack {
> >   attribute DOMString kind;
> >   attribute DOMString language;
> >   readonly attribute SourceBuffer sourceBuffer;
> > }
> >
> > partial interface SourceBuffer {
> >   readonly attribute AudioTrackList audioTracks;
> >   readonly attribute VideoTrackList videoTracks;
> >   readonly attribute TextTrackList textTracks;
> > }
> >
> > I think this will make accessing/modifying this information more natural
> and maintains the goal of not depending on a specific version of the HTML
> spec. This simply specifies the requirements for implementations that use
> MSE. If HTML.next decides to include the new declarations for xxx.kind and
> xxx.language then that is fine as long as the behavior is consistent
> between the 2 specs. This shouldn't be a huge problem since only setter
> behavior is being defined and editors of the 2 specs can coordinate on the
> definition.
> >
> > Obviously I don't want this to derail the FPWD. That ship has sailed,
> but it may be a good idea to file a bug and update the FPWD candidate with
> a link to this proposal.
> >
> > What do people think about this?
> >
> > Aaron
>
>