Re: Acessibility of <audio> and <video>

Lachlan Hunt 2008-09-01 23.36:

> Leif Halvard Silli wrote:
>> Justin James 2008-09-01 19.42:
>>> I can imagine the furor if we also applied this logic to
>>> images, by saying, "if you want accessible images, use a format
>>> that natively supports metadata of alternate text, or put a
>>> subtitle/caption/legend/etc. in your image."
> 
> No. Unlike video, images have no way embedding accessibility features 
> within them, and their meaning is very often depending on the context. 
> However, videos do have various ways of including native accessibility 
> features, such as subtitles/captions or audio descriptions.


Our recent debate about EXIF proves opposite.

Just like you said that accessible video should be the video 
editor's task (not the web page editor), one could similarely hand 
the task of photo fallback to the photo editor.

>> Add to that that movie formats are *very* often used for displaying 
>> photos.
> 
> Could you elaborate on what you mean, provide some sort of evidence to 
> support this claim, and also explain why it's relevant?

Dagbladet.no publishes vidoes which serves both a photo role and a 
video role almost daily. Here is one article from yesterday:

Link: http://www.dagbladet.no/kultur/2008/09/01/545443.html

One could have taken away the video functionality, and it would 
still be a photo. (I, for one, did not care to look at the video.)

This is a news article. Other uses cases could be news from sports 
events: The video offers you a choice between seeing the photo 
which documents the winner of the event, or you can play the video 
to see how it all happened.

>> Plus the fact the <video> elment itself has a poster image to be 
>> displayed before the video is started.
> 
> I'm not sure why that is relevant.  The poster frame can be thought of 
> as being more like an icon representing the video.  It's the content of 
> the video that is important

The Dagbladet.no example I gave above shows that the poster image 
  can serve the role of a photo.

If the poster image shows Putin looking at a sleeping siberian 
tiger, then why do you need a separate <img> with the same motif?

And just as I was more than satisfied from seeing the photo, 
likewise could anyone incapable of seeing the video be more 
satisfied with an @alt text for the poster photo. (Rather than 
having to be bothered with listening/seeing the entire video 
before getting any clue.)

This has me asking: Why couldn't the poster image just be a 
regular <img> element, rather than an attribute?

>> instead allow textuall fallback <element>inside the element</>, then 
>> that would be better.
> 
> OK, let's look at the various kind of alternatives that could be 
> potentially provided for people who either can't watch or don't want to 
> watch the video.

 >

> * Transcript of spoken content.
> * Textual descriptions of relavant non-spoken content.
>   e.g. descriptions of significant actions or sounds in the video.
> * Still images illustrating significant moments from the video.
>   e.g. images of presentation slides, if the video was of someone giving
>   a presentation.


"Still images illustrating significan moments" - that is exactly 
the usecase we have in the Dagbladet.no newsarticle I sited.

The poster image should be viewed as <video> fallback!

Because, as it turns out, almost all humans prefers an 
"image-fallback" in form of a poster, rather than have the video 
start playing once the page has loaded.

It is when looking at the fallback, that we should be offered the 
choice between how we want the video served: as slides, transcript 
or the very video.


> * A link to download the video, possibly in alternative formats, to
>   watch in an external media player, perhaps in several formats.


When it comes to getting the video in another video format, then - 
if we are talking /fallback/ - this requires that the user /knows/ 
that he does not want the default format - and that he also knows 
that he doesn't want any of the (textual) fallback either.

> * Video embedded in the page using an alternative method
>   e.g. Flash, the Cortado Theora applet, or whatever
> 
> Any or all of those could be provided and all of them would be suitable 
> for people in the following categories:
> 
> 1. People using browsers that don't support <video>
> 2. People using browsers that do support video, but don't support the
>    codecs used.
> 3. People who can't see the video well (blind or visually impaired)
> 4. People who can't hear the video well (hearing impared or no
>    headphones/speakers available)
> 5. People who want to search the content of the video
>    e.g. Instead of seeking through the video to find the information
>    they want, just look through the transcript, from which they can also
>    copy quotes if they want.
> 
> Given that the alternative content is useful to so many people, 
> regardless of physical disability or technological limitations, it makes 
>  sense for it to be provided in a way that makes it available to 
> everyone.  This is one reason why hiding alternative content away within 
> the video element is not helpful because it only makes it readily 
> available to a small subset of those people who might want or need it.

I very much agree that the way e.g. the <object> element tends to 
work is not ideal, as it does not allow me to select one variant 
over another.

This is actually also why it could be suitable to use an <img> 
instead of a poster attribute, for the video element.

> Another reason to consider is that we know very well what authors do 
> when they are encouraged to hide alternative content specifically for 
> accessibility within elements.  For example:
> 
> <noframes>Your browser does not support frames, Please upgrade to
> Netscape 4</noframes>
> 
> <noscript>Sorry, your browser doesn't support javascript.</noscript>
> 
> <object ...>Please install flash</object>

Absolutely. A problem.

> It might work ok for <video> during the transitional period when not all
> browsers implement it, since authors will gain practical benefits by
> including alternative embedding mechanisms within the video element. e.g.
> 
> <video src="movie"><object data="movie"></object></video>
> <p><a href="movie">Download movie</a>
>    <a href="transcript">Read transcript</a> ...
> 
> But, in the long term, after browsers with native video support are more
> widely used, we'll likely start seeing people leaving the video element
> empty or including useless messages, and accessibility loses.  Whereas 
> by encouraging people to use visible alternative content,
> everybody wins.

I think you limit the options. The reason why Dagbladet.no 
combines the role of photo and video is un order to save space.

Anne mentioned using SMIL to provide different media variants to 
different audiences. I am not familiar with SMIL.

But as Ann IMHO hinted, I think that the problem you are 
describing needs to be taken care of in a "microformat manner".

So, take that poster image again. If instead of building the 
poster image into the <video> element, one cold design the <video> 
element so that one may use the <img> element in order to display 
a poster image, then there would be no problem with fallback for 
the poster image, as the <img> would provide that fallback.

Let us turn things upside down: Let's say that when the users 
enters a page containing a <video>, then what he should be served 
first, is the /fallback/ of that video. That fallback would either 
be the <img> element, or it would be the @alt text of that <img>. 
(Of course, it could be an image map also.)

That way we would also solve another problem: The fallback would 
not be hidden, since the <img> would actually be fallback for the 
video. And /then/ one could click the video button to run the 
video - or possible other media alternatives to the video.

I think such an approach should be inline with an augmentative 
authoring approach.
-- 
leif halvard silli

Received on Monday, 1 September 2008 23:52:00 UTC