Re: [MSE] Questions about setting track language & kind (Bug 17006)

From: Aaron Colwell <acolwell@google.com<mailto:acolwell@google.com>>
Date: Friday, September 21, 2012 12:09 PM
To: "<public-html-media@w3.org<mailto:public-html-media@w3.org>>" <public-html-media@w3.org<mailto:public-html-media@w3.org>>
Subject: [MSE] Questions about setting track language & kind (Bug 17006)
Resent-From: <public-html-media@w3.org<mailto:public-html-media@w3.org>>
Resent-Date: Friday, September 21, 2012 12:10 PM

Hi,

On one of the calls several weeks ago I said I'd start a thread about several questions I had about Bug 17006<https://www.w3.org/Bugs/Public/show_bug.cgi?id=17006>. Here it is. :) The goal of this bug is to provide a way to reflect the role & language specified in a DASH manifest in the {Audio | Video | Text}Track objects.

I've spent some time trying to understand the DASH spec and have come up with these questions:

1. Do people still want this feature? I believe it was one of the open issues our friends at Microsoft asked to be included in the original proposal.

There are specifications that define the use DASH MPD descriptors for conveying video, audio and text track meta data used for setting the equivalent attributes in HTML5 objects. I think this feature is required.


2. Why would it be better to put this information in the manifest instead of the initialization segments? Don't they have role & language information encoded in them?

Couple of reasons. Putting the information in the manifest allows for a common representation that can be independent of which DASH profile (ISO BMFF, MPEG-2 TS or some future profile (WebM?)) is used. Also, as cited earlier, there are specifications that define use of the manifest for this information.


3. It looks like language & role can be specified at the AdaptationSet & ContentComponent level. How should these be treated differently in the Media Source context?

Not sure what the question is but use of these attributes in the ContentComponent override their use in the AdaptationSet.


4. In the context of this bug, are we assuming a 1:1 mapping between AdaptationSets and SourceBuffers?

I think that is the implications of MSE spec section 2.4 bullet 1.

(ie Representations from different AdaptationSets won't be mixed)

5. Are contentComponent id's specified in SubRepresentations  required to have the same language and role across all Representations in an AdaptationSet?

This is an AdaptationSet requirement.

If not, I believe this could mean the language for tracks could change if the application switches representations in an adaptation set.

6. There don't appear to be trackIDs mentioned in the manifest. Is it safe to assume that role & language apply to all tracks within the representation?
If so, how are alternate language tracks represented within an AdaptationSet?

An AdaptationSet (or ContentComponent in an AdaptionSet) is equivalent to a audio, video or text track, so role and language apply at that level. Alternate language tracks are different AdaptationSets in a Period. Section G.1 in the Dash spec shows an example of this.


7. What is the expected behavior if the language of a track suddenly changes?

I think this would equate to a new Period (with different AdaptationSets). The MSE spec section 2.4 bullet 1 states that the number and type of tracks needs to constant in a source buffer. So wouldn't this scenario result in a new source buffer?

Say I have 2 audio tracks. Track 1 is English and track 2 is French. My preferred language is English so track 1 is selected. I then append a new initialization segment that indicates track 1 has French and track 2 is English along with a few media segments.

Doesn't this violate the MSE 2.4 bullet 1?


  a. Should the UA switch to track 2 at the appropriate time so that English audio is always output?
  b. Should this kind of language change be rejected?


Aaron