RE: text track associations

Interesting approach. It's a somewhat misnamed use of a video element to supply audio and captions but not video. Audio files don't work today in two of the major browsers; and It's also far from clear to me that it the spec currently allows it, but if you say as editor it does, then I guess it could be made to work. 

I think we will need to change the spec to indicate more clearly that the <video> element is supposed to work if there is no video data supplied.

For example, change:
"A video element is used for playing videos or movies."
To
"A video element is used for playing videos or movies or audio".

And  replacing
"The video element is a media element whose media data is ostensibly video data, possibly with associated audio data"

With
"The video element is a media element whose media data is ostensibly video data, audio data or possibly video data with associated audio data."

I'm also not clear if the section on "Media elements", which is indicated to 'apply equally to video and audio', means that if I supply a video to an audio element; that is supposed to play the audio data from it? Can I create a display rectangle with CSS for an audio element to display video data?

One interesting side effect of this approach is that it gives the ability to make free standing timed text elements, since I can just do <video src=24hrsOfSilence.mp3> <track src=captions.cues ></video>. Would be nicer to have a shorthand <cues> element for that, but I guess not much point trying to make HTML logical at this stage.

-----Original Message-----
From: Ian Hickson [mailto:ian@hixie.ch] 
Sent: 12 April 2011 01:03
To: Sean Hayes
Cc: Silvia Pfeiffer; Mark Watson; Eric Carlson; public-html-a11y@w3.org
Subject: Re: text track associations

On Fri, 8 Apr 2011, Sean Hayes wrote:
>
> In a multitrack API, since caption tracks are essentially the audio for 
> Deaf and HoH use, we will need some way for these to follow the selected 
> audio asset.
> 
> If I have a presentation, with the option of switching to different a 
> different audio resource (e.g director commentary which is longer, 
> starts before and ends after the main movie). The captions for it need 
> to be timed to the commentary audio, and so if it's in a markup element, 
> live in that element. If the user has captions turned on and wants the 
> commentary audio, then the captions for the commentary soundtrack will 
> need to replace the captions for the original audio track, and render 
> into the rectangle allotted for the main video.
> 
> I'm not seeing how I would create this.

Use <video> element for the commentary audio, with the appropriate 
captions specified, and layer it on top of the main <video>.

-- 
Ian Hickson               U+1047E                )\._.,--....,'``.    fL
http://ln.hixie.ch/       U+263A                /,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'

Received on Wednesday, 13 April 2011 13:41:41 UTC