Re: Tech Discussions on the Multitrack Media (issue-152)

On Feb 10, 2011, at 3:08 PM, Silvia Pfeiffer wrote:

> Hi Mark,
> 
> On Fri, Feb 11, 2011 at 12:47 AM, Mark Watson <watsonm@netflix.com> wrote:
> 
> 
>> The distinction between TextTrack and MediaTrack in the API under option (1)
>> seems strange to me. Text is just another kind of media, so shouldn't the
>> kind for each track be ( Audio | Video | Text ) rather than ( Media | Text )
>> where Media = ( Audio | Video ) ? [This is how it is framed in option 3,
>> albeit up one level].
> 
> That is indeed one of the key questions we will have to answer: do we
> treat media tracks differently from text tracks. One the one hand they
> can all abstractly be regarded as "some time-aligned data for a
> video". On the other hand, the current specification of <track> does
> not allow child elements and the TimedTrack API is very specifically
> targeted at text tracks: the concept of text cues does not make sense
> for media tracks. Therefore it was necessary to introduce MediaTrack
> as an additional API. But indeed one of the questions we need to
> answer is whether that is a good idea or not.
> 

Would it be possible to define a MediaTrack class for the common stuff and re-define the existing TextTrack class to inherit from that (adding the cues stuff) ?. Or is it too late to make that kind of change to the TextTrack ?

> 
>> If I read rightly this condition is not met in
>> (3), (4) and (6) right ?
> 
> That's not quite right. The JavaScript API is still the same for both
> in-band and external media tracks in most of these cases.
> 
> In case (3) we have AudioTrack and VideoTrack, which apply equally to
> in-band and external media tracks.

Ah, ok - so is it a general assumption that whatever kind of objects are created according to the markup proposal, the same kind of objects are also added to the tracks list for in-band tracks (as is specified for text tracks) ? 

I was wondering whether it made sense for in-band tracks to result in a complete HTMLMediaElement object being created in the tracks list: it seems you just need a much more limited object like TextTrack, with just the kind/label/language and mode.

...Mark


> 
> You are correct for case (4) though: the synchronization entity moves
> from the main video to the par above it, such that in-band tracks
> cannot be supported like this any longer, since in-band tracks are not
> associated with a parent element. I agree that (4) is not an
> acceptable solution to this problem. It exists more for historic
> purposes, since the question of re-using SMIL constructs keeps coming
> up and the markup seemed to make sense. But the JavaScript API for
> this just cannot be made consistent.
> 
> Approach (6) however would again work the same for in-band and
> external media. As long as we include the gathering of the dependent
> media elements into browser code, we can also add to this set the
> in-band tracks and thus make all tracks uniformly available through
> something like video.audioTracks and video.videoTracks.
> 
> Cheers,
> Silvia.
>

Received on Friday, 11 February 2011 21:11:19 UTC