Re: [MSE] Resolving Bug 24370

On Mon, May 12, 2014 at 11:01 PM, Aaron Colwell <acolwell@google.com> wrote:
> On Sun, May 11, 2014 at 12:37 PM, Silvia Pfeiffer
> <silviapfeiffer1@gmail.com> wrote:
>>
>> On Thu, Apr 3, 2014 at 5:30 AM, Aaron Colwell <acolwell@google.com> wrote:
>> >
>> >
>> > On Wed, Apr 2, 2014 at 6:20 AM, Jim Ley <jim.ley@gmail.com> wrote:
>> >>
>> >> On 2 April 2014 09:26, Silvia Pfeiffer <silviapfeiffer1@gmail.com>
>> >> wrote:
>> >>>
>> >>> On Wed, Apr 2, 2014 at 6:04 PM, Jim Ley <jim.ley@gmail.com> wrote:
>> >>> >
>> >>> > Isn't the use case for language being mutable simply live subtitles
>> >>> > of
>> >>> > a
>> >>> > live "channel" that has programmes in different languages and
>> >>> > therefore
>> >>> > has
>> >>> > closed captions / subtitles in different languages.
>> >>>
>> >>> Each one of those subtitle languages would be a different track, so
>> >>> there is no need for mutability.
>> >>
>> >>
>> >> No they're not, some shows are in English, some are in Welsh, the
>> >> subtitle
>> >> track is a single one for the live broadcast, similarly with audio, the
>> >> broadcast audio changes languages.
>> >
>> >
>> > I think there are 2 separate issues here.
>> > 1. How are language changes within an inband track supposed to be
>> > represented in HTML5?
>>
>> Do you think we should mention language changes as another example
>> where multiple tracks are created in
>>
>> http://rawgit.com/silviapfeiffer/HTMLSourcingInbandTracks/master/index.html
>> ? Would that help clarify?
>
>
> I was viewing that document as providing container specific mappings to
> HTML5 concepts.

Correct.

> It seems to me that if HTML5 expects xxxTrack objects to be
> removed and readded when language and kind changes, then I feel like that
> should be in the HTML5 spec.

The HTML5 spec has a single kind and language associated with a track
for its lifetime. It does not deal with such changes and that's on
purpose. We're still discussing adding a note about the effect of
attribute changes in
https://www.w3.org/Bugs/Public/show_bug.cgi?id=24563 which should help
clarify this.


> At a minimum I think it needs to provide
> clarity about when and under what conditions xxxTrack objects are expected
> to come and go throughout the life of the presentation. I don't think this
> is clear especially for presentations where there are language changes or
> the addition and removal of tracks in the middle of the presentation like a
> SAP track coming and going whenever there is a change between a program and
> a commercial.

I assume SAP = secondary audio program, so it's a secondary audio track?

I guess that all depends on your definition of a "track".

The HTML spec already says:
"A media resource can have multiple audio and video tracks. For the
purposes of a media element, the video data of the media resource is
only that of the currently selected track (if any) given by the
element's videoTracks attribute, and the audio data of the media
resource is the result of mixing all the currently enabled tracks (if
any) given by the element's audioTracks attribute."
and
"If a media resource dynamically adds or removes audio or video
tracks, then the indices of the tracks will change dynamically. If the
media resource changes entirely, then all the previous tracks will be
removed and replaced with new tracks."

You're right, though, in that it doesn't explicitly state what it
expects semantically to be in those tracks, even though it provides
examples in the section where AudioTrackList and VideoTrackList
objects are definted. You could register a bug to clarify this in a
note. IMO since the change of the kind and language attributes has no
effect, it seems clear that a track semantically only ever has one
kind and one language.

As for the mapping of in-band tracks - the specifics of non-HTML file
formats are somewhat outside the concern of HTML, so I think the
separate spec makes a lot of sense to explain what happens when a file
format that doesn't match the expectations of the HTML spec needs to
be mapped into it.


> It also isn't clear to me whether seeking should cause certain
> tracks to disappear/reappear if they aren't available in that portion of the
> timeline. I don't feel like either of these things are specified very well
> in the current HTML5 spec text.

Right - I guess that's a quality of implementation detail. As long as
the track exists when the Web page needs it, the rest is about memory
management. We don't expect a full media resource to always be
buffered in the browser - that's what buffered and seekable TimeRanges
are for. That applies to data in audio, video and text tracks. If
there are any shortcomings in describing how this works, you should
register a bug.


> On a related note, I think that the HTML5 spec should also mention that
> multiple xxxTrack objects should be created if a track in the underlying
> container has multiple kinds or languages associated with it.

That could be a note in the section on "multiple media tracks, though
as I said it seemed obvious to me because of the way the attributes
are dealt with. Register a bug? (sorry that that's my standard
answer).

> It was not
> obvious to me that this was the intended behavior for this situation and I
> think it deserves a sentence or 2 in the HTML5 spec. I think the HTML5 spec
> text implies there is a 1-1 mapping between xxxTracks objects and tracks in
> the container formats, but the discussions around language/kind changes,
> multiple kinds, and things like VBI captions in a container level video
> track make me believe that this is actually not true.

Actually, it simply states nothing to that effect, leaving it
completely to the UA to sort out.


> All of these
> situations result in multiple HTML5 tracks, according to my current
> understanding, but only actually constitute a single track at the container
> level. I'm not aware of any place in the HTML5 spec that these nuances are
> covered.

I'll improve the explanation of this in the in-band track mapping
spec. For all other informative notes in the HTML spec, do register a
bug.

HTH,
Silvia.

Received on Monday, 19 May 2014 02:25:18 UTC