Re: Tech Discussions on the Multitrack Media (issue-152)

On Mar 1, 2011, at 11:58 PM, Philip Jägenstedt wrote:

> On Tue, 01 Mar 2011 23:51:30 +0100, David Singer <singer@apple.com> wrote:
> 
>> Hi Silvia, friends
>> 
>> Eric and I have been discussing this, and we've added an 8th option to  
>> the Wiki, for your consideration.
>> 
>> at <http://www.w3.org/WAI/PF/HTML/wiki/Media_Multitrack_Media_API>
> 
> This is like option 2 except that <track> is not a void element and can  
> have <source> child elements, right?
> 
> For all of the options that don't use <audio> and <video> to represent  
> audio and video tracks, do you consider it a problem that properties of  
> HTMLMediaElement are not available? Some of these allow for interesting  
> conflicts such as looping a slave video but not the master video, but I  
> imagine at least videoWidth, videoHeight and volume would be useful to  
> have.

It seems that the requirements for the object representing a track are more than the proposed MediaTrack but less than HTMLMediaElement.

If something other than strict time alignment in the playout of the different tracks is required, then shouldn't you have completely separate media elements, rather than allowing currentTime, playbackRate, loop attributes & load(), play(), pause() methods for each track ? It could surely get quite complex if each track is allowed to have independent timing, but somehow they are still supposed to be linked together.

What attributes from HTMLMediaElement would really make sense to add to MediaTrack ?

I have another question related to an earlier discussion on this thread: do I understand correctly that in order to apply CSS styling there needs to be an HTML markup element to apply the styling to ? i.e. that you couldn't apply styling to control the rendering of an in-band track that appears only as a MediaTrack object and not as a <track> element (with associated HTMLTrackElement object) ?

In that case it seems we need to be able to refer to an in-band track in the main resource from a <track> element in order to apply styling. For example the MediaTrack API could expose an "id" attribute (at least for in-band tracks) which could be used to refer to this track from a <track> element.

> 
> About multiple text track formats, my working assumption is that we will  
> be able to agree on a single format that everyone supports as there is not  
> really any of the legal of business issues that we have for audio and  
> video.

Even if all browsers, including those embedded in TVs etc., support WebVTT, it seems likely some will support other formats as well (e.g. TTML/SMPTE-TT) and that content providers might have a preference about which is used on devices which support both. 

> 
> If, however, we should both fail at finding a baseline text format and  
> decide that we should use <track> for audio and video as well, then  
> wouldn't multiple <track> elements with a type parameter be enough? We  
> could define it such that multiple <track> elements that differ only in  
> type should be assumed to representing the same content and have some kind  
> of track selection algorithm for picking one. Am I too optimistic?
> 

That seems like a secondary markup language design issue. <source> elements does allow you to express the semantics that some things are mutually exclusive. The version with <source> elements inside <track> is a superset of the version with only <track> elements as you can always list multiple <track> elements instead. I would be cautious about being too prescriptive with a "pick one" algorithm: if the tracks are unambiguously described and the browser has a good way to make use of two of the same type and that is what the user wants then it should be allowed to do that.

...Mark

> -- 
> Philip Jägenstedt
> Core Developer
> Opera Software
> 
>

Received on Wednesday, 2 March 2011 16:41:06 UTC