Re: Tech Discussions on the Multitrack Media (issue-152)

Hi everyone,

I have a couple of comments on this proposal, but first, since this is my first post to this list I should introduce myself. I am representing Netflix - we joined W3C just this week. We are interested in ensuring that a streaming service like ours could in future be supported by HTML5.

One thing we are interested in is support for multiple languages, for both audio and text tracks and therefore a Javascript API to discover and select amongst those tracks.

We use a form of HTTP adaptive streaming, which if translated to HTML5 would mean providing a URL for a manifest to the <video> element as in Option (1) on the wiki page. But there is also another case where there are multiple tracks and no HTML markup: when there are multiple tracks inside a single ordinary media file e.g. an mp4 file with multiple audio language tracks.

The distinction between TextTrack and MediaTrack in the API under option (1) seems strange to me. Text is just another kind of media, so shouldn't the kind for each track be ( Audio | Video | Text ) rather than ( Media | Text ) where Media = ( Audio | Video ) ? [This is how it is framed in option 3, albeit up one level].

I don't have a strong opinion on the markup aspect, but I think the first side-condition is important (that the API be the same whether the tracks come from explicit markup or are present within a multiplexed file or described by a manifest). If I read rightly this condition is not met in (3), (4) and (6) right ?

Best,

Mark Watson
watsonm@netflix.com<mailto:watsonm@netflix.com>
On Feb 9, 2011, at 11:56 PM, Silvia Pfeiffer wrote:

Everyone,

Your input on this is requested.

Issue-152 is asking for change proposals for a solution for media
resources that have more than just one audio and one video track
associated with them. The spec addresses this need for text tracks
such as captions and subtitles only [1]. But we haven't solved this
problem for additional audio and video tracks such as audio
descriptions, sign language video, and dubbed audio tracks.

In the accessibility task force we have discussed different options
over the last months. However, the number of people that provide
technical input on issues related to media in the TF is fairly
limited, so we have decided to use the available time until a change
proposal for issue-152 is due (21st February [2]) to open the
discussion to the larger HTML working group with the hope of hearing
more opinions.

Past accessibility task force discussions [3][4] have exposed a number
of possible markup/API solutions.

The different approaches are listed at
http://www.w3.org/WAI/PF/HTML/wiki/Media_Multitrack_Media_API . This
may be an incomplete list, but it's a start. If you have any better
ideas, do speak up.

Which approach do people favor and why?

Cheers,
Silvia.

[1] http://www.whatwg.org/specs/web-apps/current-work/multipage/video.html#the-track-element
[2] http://lists.w3.org/Archives/Public/public-html/2011Jan/0198.html
[3] http://lists.w3.org/Archives/Public/public-html-a11y/2010Oct/0520.html
[4] http://lists.w3.org/Archives/Public/public-html-a11y/2011Feb/0057.html