Re: Acessibility of <audio> and <video>

On Sep 5, 2008, at 12:02 AM, Henri Sivonen wrote:

> On Sep 4, 2008, at 18:29, Eric Carlson wrote:
>
>> On Sep 4, 2008, at 1:17 AM, Henri Sivonen wrote:
>>
>>> On Sep 4, 2008, at 01:13, Dave Singer wrote:
>>>
>>>> 2.1.2 Configuring
>>>> Sometimes, similarly, the media format itself can carry optional  
>>>> features. An example might be the 3GPP file format (or any file  
>>>> format from that family, such as MP4) with a text track in 3GPP  
>>>> Timed Text format. Enabling this track (and thereby causing it to  
>>>> be presented) may be a way to satisfy a need within a single  
>>>> media file.
>>>
>>> It seems to me that for captioning, an off-by-default track within  
>>> the main file is preferable over burned-in open captions, because  
>>> tracks within the main file travel better, compress better (and  
>>> transferring the captions even when not needed is not burdensome  
>>> in terms of relative network bandwidth) and make video more  
>>> searchable.
>>>
>> I am not sure if you are suggesting otherwise, but a a 3GPP Timed  
>> Text track is exactly what you describe: a relatively small text  
>> track carried within the media file. It may nor may not be enabled  
>> by default, that is a decision that is made at authoring time.
>
> Yes, I mean 3GPP Timed Text in the MP4 context. Possibly Kate in an  
> Ogg context, but that isn't sorted out yet.
>
> What kind of metadata about captions vs. subtitles, on-by-default  
> vs. off-by-default and language can MP4 contain about an 3GPP track?  
> Does QuickTime expose in the API whatever the file format can  
> express here?
>
   A 3GPP text track is just a text track. It can be used for "sub  
titles" or "close caption" text, but that is up to the media producer  
and/or consumer. Any type of track can be enabled or disabled by  
default, and can be tagged with a language code. QuickTime movie  
(.mov) and Apple MPEG-4 (.m4v) can have Closed Caption tracks, which  
carry CEA608 data with timing, style, screen position, etc information.

   QuickTime has APIs to query and/or change track attributes at  
runtime.

>>>> (In some cases, the media format may also need to disable a  
>>>> track; for example, a track providing audio description of video  
>>>> may incorporate the standard audio within it, and the normal  
>>>> audio track would be disabled if the audio description were  
>>>> enabled.)
>>>
>>> I would guess that content providers would opt for alternative  
>>> files in this case, because additional audio tracks show up on the  
>>> bandwidth bill if served even when not needed.
>>>
>> This is not necessarily true. Even for progressive download files,  
>> some media-subsystems only read the parts of a file necessary for  
>> the presentation.
>
> How does this work?
>
   In several different ways:

  - In some contexts, the QuickTime IO sub-system downloads media data  
only when it is requested by the media handling sub-system (allowing  
for latency of course), so only data that will be presented is loaded.

  - A QuickTime movies doesn't hae to be self contained, but can  
reference media in external files. Even very old versions of the  
QuickTime browser plug-in (circa 1997) don't download data from  
external files unless it is in an enabled track.

  - Media data in streamed movies (eg. rtsp) is never downloaded until  
it is needed.


>>>> We therefore also need the ability to apply the same preferences  
>>>> used for selection, to configuring the file. Note that not all  
>>>> media sub-systems will offer the user-agent such an API; that is  
>>>> acceptable – for media files associated with those systems, the  
>>>> files are not configurable and selection must be used instead.
>>>
>>> This seems alarming. Does at least one of QuickTime, GStreamer or  
>>> DirectShow lack such an API? If one of those lacks such an API,  
>>> can such an API be put in place in a timely manner?
>>>
>>> It seems to me that if automatic selection isn't reliable, content  
>>> providers will shy away from an automatic selection system.
>>>
>> I believe David is pointing out that not all sub-systems have this  
>> capability to emphasize that is it crucial for content authors to  
>> be able to structure their markup so one of several files is  
>> selected based on the user's stated preferences.
>
> QuickTime, GStreamer and DirectShow seem to be the subsystems that  
> make or break the proposal. Is this a problem with those three?
>
   I don't understand how the (in)abilities of these sub-systems make  
or break the proposal. David's proposal allows for sub-system that  
support alternates within a media file as well as those that do not.  
If a media format/syb-system does not allow alternates, the content  
author using that format can create alternate files and instruct the  
UA to select among them with alternate <source> elements.

Eric Carlson
Apple

Received on Friday, 5 September 2008 23:01:44 UTC