Re: Tech Discussions on the Multitrack Media (issue-152)

Hi Philip,

Just a quick note that the "alternative" vs "additional" distinction is not always completely clear. Video with different camera angles (gimmiky or not) could be considered as an alternative, or could be rendered as picture-in-picture, or multiple thumbnail videos could show beside the main video (some sports sites already do this kind of thing).

If you have the output capabilities (e.g. wireless headphones plus regular speakers) then simultaneous output of different audio languages might make sense.

Not to say these are compelling use-cases, just that the markup should indicate what the media actually is such that the player can decide what to do, without any hard "alternative" vs "additional" distinction.

The one case where I see a clear distinction is with audio description tracks which may or may not include the original audio.

It might make sense to be able to label tracks as "additional", meaning they have no "stand-alone" value, but the absence of this label does not mean that the track cannot be played in combination with other tracks if this makes sense based on the nature of the track.

...Mark

On Feb 15, 2011, at 7:49 AM, Philip Jägenstedt wrote:

> On Thu, 10 Feb 2011 00:56:19 +0100, Silvia Pfeiffer  
> <silviapfeiffer1@gmail.com> wrote:
> 
>> Everyone,
>> 
>> Your input on this is requested.
>> 
>> Issue-152 is asking for change proposals for a solution for media
>> resources that have more than just one audio and one video track
>> associated with them. The spec addresses this need for text tracks
>> such as captions and subtitles only [1]. But we haven't solved this
>> problem for additional audio and video tracks such as audio
>> descriptions, sign language video, and dubbed audio tracks.
>> 
>> In the accessibility task force we have discussed different options
>> over the last months. However, the number of people that provide
>> technical input on issues related to media in the TF is fairly
>> limited, so we have decided to use the available time until a change
>> proposal for issue-152 is due (21st February [2]) to open the
>> discussion to the larger HTML working group with the hope of hearing
>> more opinions.
>> 
>> Past accessibility task force discussions [3][4] have exposed a number
>> of possible markup/API solutions.
>> 
>> The different approaches are listed at
>> http://www.w3.org/WAI/PF/HTML/wiki/Media_Multitrack_Media_API . This
>> may be an incomplete list, but it's a start. If you have any better
>> ideas, do speak up.
>> 
>> Which approach do people favor and why?
>> 
>> Cheers,
>> Silvia.
>> 
>> [1]  
>> http://www.whatwg.org/specs/web-apps/current-work/multipage/video.html#the-track-element
>> [2] http://lists.w3.org/Archives/Public/public-html/2011Jan/0198.html
>> [3]  
>> http://lists.w3.org/Archives/Public/public-html-a11y/2010Oct/0520.html
>> [4]  
>> http://lists.w3.org/Archives/Public/public-html-a11y/2011Feb/0057.html
> 
> One aspect that isn't fully explored here is the distinction between  
> additional and alternative tracks. This is much like with text tracks, in  
> that sometimes you have tracks that make sense together and sometimes they  
> are supposed to be alternative.
> 
> Additional track: E.g. audio descriptions that are played in  
> synchronization with the original audio track.
> 
> Alternative track: E.g. a different language dub or a different angle  
> video track.
> 
> This isn't a popularity contest, but I don't favor option 1, 3, 4 or 5.  
> These remain:
> 
> Option 2 is quite sane for additional and alternative audio tracks and  
> would mostly work for alternative video tracks. However, it doesn't work  
> at all for additional video tracks that should be presented on top of the  
> other video or side-by-side.
> 
> Option 6 is a good fit for additional video tracks, but certainly not for  
> alternative video tracks. It's kind of OK for audio tracks, but would be  
> weird with <audio controls>. One disadvantage not mentioned is that it  
> requires resolving reference loops if multiple audio/video elements claim  
> to be the master timelines of each other.
> 
> Option 7 is mostly the same as 6, including the problem of reference  
> loops, it's just the syntax for binding together tracks that's different.
> 
> IMO audio tracks are really quite similar to text tracks and could be  
> solved in a similar way, it is mostly video tracks that make things messy.  
> Alternative video tracks like different angles are, IMO, mostly gimmicky  
> and not of great utility. Are there important use cases for it? Additional  
> video tracks are more obviously useful, in particular for sign language  
> overlays.
> 
> No real conclusions for me, just rambling...
> 
> -- 
> Philip Jägenstedt
> Core Developer
> Opera Software
> 
>

Received on Tuesday, 15 February 2011 18:40:41 UTC