Re: Format Requirements for Text Audio Descriptions (was Re: HTML5 TF from my team)

One comment inline.
Geoff/NCAM

On 5/4/10 8:11 PM, "Silvia Pfeiffer" <silviapfeiffer1@gmail.com> wrote:

Hi Masatomo,

Thank you very much for this feedback. It is indeed very valuable and
highly appreciated.

I have some questions inline below.


2010/5/5 Masatomo Kobayashi <MSTM@jp.ibm.com>:
>
> At this time, the additional requirements based on our research will
> include:
>   a) behavior when overflowing
>   b) extended audio descriptions
>   c) support of SSML
> If you have already discussed these topics, I would appreciate if you could
> send me any related links.
>
>
>
> a) Behavior when overflowing
>
> The current proposals seem not to explicitly mention the case in which the
> screen reader does not finish reading out a description sentence by the 'end
> time'. This is likely to be caused by at least three reasons:
> - A typical author of textual audio descriptions does not have a screen
> reader. This means s/he cannot check if the sentence is fit within the time
> frame. Even if s/he has a screen reader, a different screen reader may take
> longer to read out the same sentence;
> - Some screen reader users (e.g., elderly and people with learning
> disabilities) may slow down the speech rate; or
> - A visually-complicated scene (e.g., figures on a blackboard in an online
> physics class) may not be sufficiently described within any time interval in
> the original audio track.
>
> So, the specification should support to specify the behavior for this case.
> The options will include:
> - none -- continue to read out the sentence even after the end time. This
> may overlap important information in the video.
> - clip -- force to stop reading out the sentence at the end time. This may
> cause the user to miss important information in the sentence.
> - extend -- pause the video at the end time until the screen reader finishes
> reading out the sentence. This may require an additional mechanism beyond
> "aria-live: assertive", but at least our prototype aiBrowser can do it.
>
> This option might be able to be specified as an attribute:
>   <track src="..." type="text/srt" role="textaudesc"
> overflow="extend"></track>
> Or using CSS like the 'overflow' property for a visual element:
>   track[role= textaudesc] {audio-overflow: extend}
>
> For now only 'textaudesc' tracks need this mechanism, but in the future
> other types of tracks, such as synthesized sign language, would need to be
> covered.


This is an interesting requirement.

Thinking about it in more depth, we may even want to use such an
attribute on captions and subtitles. It would indicate what will
happen if caption elements overlap into the next caption text cue, ie.
just display both (which would be the default) or clip the cue.
Pausing the video probably doesn't make sense for caption text.

I like this attribute.

GF:
I agree that this would be useful.  I'm not sure if I'm misinterpreting your comment about captions, but I do think that having the ability to pause video/audio tracks to allow for supplemental captions would be a good idea.  As there are cases for using extended audio descriptions to deliver extra information, there are cases for extended captions to do the same:  for example, a user could elect to view an extended caption track so that during a complex (pre-recorded) chemistry lecture, the video and audio pause in order to accommodate explanatory text.

Geoff/NCAM