Re: Tech Discussions on the Multitrack Media (issue-152)

On Tue, Mar 1, 2011 at 5:46 AM, David Singer <singer@apple.com> wrote:
>
> On Feb 27, 2011, at 16:39 , Silvia Pfeiffer wrote:
>
>> For both of the below to work, the browser has to implement an
>> extraction of the cues from the binary media resource into a TextTrack
>> with TextTrackCues, which then exposes it to the browser and to
>> JavaScript. Since by default only TextTracks of @kind=subtitles or
>> @kind=captions are rendered and these can only contain raw text, it
>> probably makes sense to declare both the content advisory descriptor
>> and the image-based caption as @kind=metadata.
>>
>
> Yes, I would have the media type of these be 'text', the coding format be something suitable, and the 'kind' (purpose, function, really) be something that indicated it's metadata of some sort.
>
> I think a general API
>
> Tell:
> * what the times at which the content of this track changes (in the case that the UA has a time-map, e.g. a complete caption or media file)
> * when the content of this track changes (i.e. a frame-by-frame callback)
> * what the content of this track is at this (possibly current) time
>
> covers the uses, doesn't it?


Yes, and I believe that is already in the spec through the TimedTrack API.

> * what the times at which the content of this track changes (in the case that the UA has a time-map, e.g. a complete caption or media file)

A TextTrackCue provides the start and end time of its cue:
http://www.whatwg.org/specs/web-apps/current-work/multipage/video.html#texttrackcue
.
It also raises events onenter and onexit.

> * when the content of this track changes (i.e. a frame-by-frame callback)

The TextTrack raises a oncuechange event:
http://www.whatwg.org/specs/web-apps/current-work/multipage/video.html#texttrack
.

> * what the content of this track is at this (possibly current) time

The TextTrack has a list of the currently active cues in the
"activeCues" field:
http://www.whatwg.org/specs/web-apps/current-work/multipage/video.html#texttrack
.


My point simply was that we are already able to support this through
the current spec - it's just a matter of the browsers exposing the
in-band tracks. And since these track types cannot be rendered as
subtitles or captions, they would necessarily be @kind=metadata.


> Bob's message talks of the information being repeated -- carouseled -- but that is just refreshing what the terminal already knew.  I don't think that those should constitute a change of content.

I agree - and I thought that's what I said in the next paragraph:

>> In the case of the content advisory descriptor, I would expect the
>> browser to indeed only expose the change of the descriptor in a new
>> cue. Anything else is just repeated information.


Cheers,
Silvia.

Received on Tuesday, 1 March 2011 21:01:24 UTC