Re: Tech Discussions on the Multitrack Media (issue-152)

On Thu, 10 Feb 2011 00:56:19 +0100, Silvia Pfeiffer  
<silviapfeiffer1@gmail.com> wrote:

> Everyone,
>
> Your input on this is requested.
>
> Issue-152 is asking for change proposals for a solution for media
> resources that have more than just one audio and one video track
> associated with them. The spec addresses this need for text tracks
> such as captions and subtitles only [1]. But we haven't solved this
> problem for additional audio and video tracks such as audio
> descriptions, sign language video, and dubbed audio tracks.
>
> In the accessibility task force we have discussed different options
> over the last months. However, the number of people that provide
> technical input on issues related to media in the TF is fairly
> limited, so we have decided to use the available time until a change
> proposal for issue-152 is due (21st February [2]) to open the
> discussion to the larger HTML working group with the hope of hearing
> more opinions.
>
> Past accessibility task force discussions [3][4] have exposed a number
> of possible markup/API solutions.
>
> The different approaches are listed at
> http://www.w3.org/WAI/PF/HTML/wiki/Media_Multitrack_Media_API . This
> may be an incomplete list, but it's a start. If you have any better
> ideas, do speak up.
>
> Which approach do people favor and why?
>
> Cheers,
> Silvia.
>
> [1]  
> http://www.whatwg.org/specs/web-apps/current-work/multipage/video.html#the-track-element
> [2] http://lists.w3.org/Archives/Public/public-html/2011Jan/0198.html
> [3]  
> http://lists.w3.org/Archives/Public/public-html-a11y/2010Oct/0520.html
> [4]  
> http://lists.w3.org/Archives/Public/public-html-a11y/2011Feb/0057.html

One aspect that isn't fully explored here is the distinction between  
additional and alternative tracks. This is much like with text tracks, in  
that sometimes you have tracks that make sense together and sometimes they  
are supposed to be alternative.

Additional track: E.g. audio descriptions that are played in  
synchronization with the original audio track.

Alternative track: E.g. a different language dub or a different angle  
video track.

This isn't a popularity contest, but I don't favor option 1, 3, 4 or 5.  
These remain:

Option 2 is quite sane for additional and alternative audio tracks and  
would mostly work for alternative video tracks. However, it doesn't work  
at all for additional video tracks that should be presented on top of the  
other video or side-by-side.

Option 6 is a good fit for additional video tracks, but certainly not for  
alternative video tracks. It's kind of OK for audio tracks, but would be  
weird with <audio controls>. One disadvantage not mentioned is that it  
requires resolving reference loops if multiple audio/video elements claim  
to be the master timelines of each other.

Option 7 is mostly the same as 6, including the problem of reference  
loops, it's just the syntax for binding together tracks that's different.

IMO audio tracks are really quite similar to text tracks and could be  
solved in a similar way, it is mostly video tracks that make things messy.  
Alternative video tracks like different angles are, IMO, mostly gimmicky  
and not of great utility. Are there important use cases for it? Additional  
video tracks are more obviously useful, in particular for sign language  
overlays.

No real conclusions for me, just rambling...

-- 
Philip Jägenstedt
Core Developer
Opera Software

Received on Tuesday, 15 February 2011 15:50:23 UTC