Re: [MSE] Resolving Bug 24370

On Mon, May 19, 2014 at 12:24 PM, Silvia Pfeiffer
<silviapfeiffer1@gmail.com> wrote:
> On Mon, May 12, 2014 at 11:01 PM, Aaron Colwell <acolwell@google.com> wrote:
>> On Sun, May 11, 2014 at 12:37 PM, Silvia Pfeiffer
>> <silviapfeiffer1@gmail.com> wrote:
>>>
>>> On Thu, Apr 3, 2014 at 5:30 AM, Aaron Colwell <acolwell@google.com> wrote:
>>> >
>>> >
>>> > On Wed, Apr 2, 2014 at 6:20 AM, Jim Ley <jim.ley@gmail.com> wrote:
>>> >>
>>> >> On 2 April 2014 09:26, Silvia Pfeiffer <silviapfeiffer1@gmail.com>
>>> >> wrote:
>>> >>>
>>> >>> On Wed, Apr 2, 2014 at 6:04 PM, Jim Ley <jim.ley@gmail.com> wrote:
>>> >>> >
>>> >>> > Isn't the use case for language being mutable simply live subtitles
>>> >>> > of
>>> >>> > a
>>> >>> > live "channel" that has programmes in different languages and
>>> >>> > therefore
>>> >>> > has
>>> >>> > closed captions / subtitles in different languages.
>>> >>>
>>> >>> Each one of those subtitle languages would be a different track, so
>>> >>> there is no need for mutability.
>>> >>
>>> >>
>>> >> No they're not, some shows are in English, some are in Welsh, the
>>> >> subtitle
>>> >> track is a single one for the live broadcast, similarly with audio, the
>>> >> broadcast audio changes languages.
>>> >
>>> >
>>> > I think there are 2 separate issues here.
>>> > 1. How are language changes within an inband track supposed to be
>>> > represented in HTML5?
>>>
>>> Do you think we should mention language changes as another example
>>> where multiple tracks are created in
>>>
>>> http://rawgit.com/silviapfeiffer/HTMLSourcingInbandTracks/master/index.html
>>> ? Would that help clarify?
>>
>>
>> I was viewing that document as providing container specific mappings to
>> HTML5 concepts.
>
> Correct.
>
>> It seems to me that if HTML5 expects xxxTrack objects to be
>> removed and readded when language and kind changes, then I feel like that
>> should be in the HTML5 spec.
>
> The HTML5 spec has a single kind and language associated with a track
> for its lifetime. It does not deal with such changes and that's on
> purpose. We're still discussing adding a note about the effect of
> attribute changes in
> https://www.w3.org/Bugs/Public/show_bug.cgi?id=24563 which should help
> clarify this.
>
>
>> At a minimum I think it needs to provide
>> clarity about when and under what conditions xxxTrack objects are expected
>> to come and go throughout the life of the presentation. I don't think this
>> is clear especially for presentations where there are language changes or
>> the addition and removal of tracks in the middle of the presentation like a
>> SAP track coming and going whenever there is a change between a program and
>> a commercial.
>
> I assume SAP = secondary audio program, so it's a secondary audio track?
>
> I guess that all depends on your definition of a "track".
>
> The HTML spec already says:
> "A media resource can have multiple audio and video tracks. For the
> purposes of a media element, the video data of the media resource is
> only that of the currently selected track (if any) given by the
> element's videoTracks attribute, and the audio data of the media
> resource is the result of mixing all the currently enabled tracks (if
> any) given by the element's audioTracks attribute."
> and
> "If a media resource dynamically adds or removes audio or video
> tracks, then the indices of the tracks will change dynamically. If the
> media resource changes entirely, then all the previous tracks will be
> removed and replaced with new tracks."
>
> You're right, though, in that it doesn't explicitly state what it
> expects semantically to be in those tracks, even though it provides
> examples in the section where AudioTrackList and VideoTrackList
> objects are definted. You could register a bug to clarify this in a
> note. IMO since the change of the kind and language attributes has no
> effect, it seems clear that a track semantically only ever has one
> kind and one language.
>
> As for the mapping of in-band tracks - the specifics of non-HTML file
> formats are somewhat outside the concern of HTML, so I think the
> separate spec makes a lot of sense to explain what happens when a file
> format that doesn't match the expectations of the HTML spec needs to
> be mapped into it.
>
>
>> It also isn't clear to me whether seeking should cause certain
>> tracks to disappear/reappear if they aren't available in that portion of the
>> timeline. I don't feel like either of these things are specified very well
>> in the current HTML5 spec text.
>
> Right - I guess that's a quality of implementation detail. As long as
> the track exists when the Web page needs it, the rest is about memory
> management. We don't expect a full media resource to always be
> buffered in the browser - that's what buffered and seekable TimeRanges
> are for. That applies to data in audio, video and text tracks. If
> there are any shortcomings in describing how this works, you should
> register a bug.
>
>
>> On a related note, I think that the HTML5 spec should also mention that
>> multiple xxxTrack objects should be created if a track in the underlying
>> container has multiple kinds or languages associated with it.
>
> That could be a note in the section on "multiple media tracks, though
> as I said it seemed obvious to me because of the way the attributes
> are dealt with. Register a bug? (sorry that that's my standard
> answer).
>
>> It was not
>> obvious to me that this was the intended behavior for this situation and I
>> think it deserves a sentence or 2 in the HTML5 spec. I think the HTML5 spec
>> text implies there is a 1-1 mapping between xxxTracks objects and tracks in
>> the container formats, but the discussions around language/kind changes,
>> multiple kinds, and things like VBI captions in a container level video
>> track make me believe that this is actually not true.
>
> Actually, it simply states nothing to that effect, leaving it
> completely to the UA to sort out.
>
>
>> All of these
>> situations result in multiple HTML5 tracks, according to my current
>> understanding, but only actually constitute a single track at the container
>> level. I'm not aware of any place in the HTML5 spec that these nuances are
>> covered.
>
> I'll improve the explanation of this in the in-band track mapping
> spec. For all other informative notes in the HTML spec, do register a
> bug.

Done. There are now more examples in the intro of the in-band track
mapping spec: http://rawgit.com/silviapfeiffer/HTMLSourcingInbandTracks/master/index.html#introduction
.

Silvia.

Received on Monday, 19 May 2014 11:50:58 UTC