Re: [media] handling multitrack audio / video

I like the idea of expanding the "kind" attribute. I came up with the same idea yesterday while discussing the pros and cons of media queries with Eric. I think media queries are a poor fit for what we need to do to describe alternatives with built-in accessibility affordances. Media queries are designed to describe the rendering context so that content can adapt. In this case, we want to describe the media resource so that the UA can choose the best version according to user preferences. kind (particularly with a variant that can distinguish optional vs. burned-in affordances, and which can list multiple affordances) seems like a much more natural fit.

I don't think the <switch> element really helps here, because the challenge is that we need best-fit, not first-fit behavior, and <switch> doesn't help with that at all relative to <source> selection.

My suggestion for auxiliary media resources is to let the <track> element take an id instead of a src which points to an <audio> or <video>. That plus a kind label on the track can provide the needed linkage without complicated changes to content models.

Regards,
Maciej

On Dec 2, 2010, at 1:28 PM, Silvia Pfeiffer wrote:

> On Fri, Dec 3, 2010 at 2:41 AM, Eric Carlson <eric.carlson@apple.com> wrote:
>> 
>> On Dec 2, 2010, at 6:31 AM, Silvia Pfeiffer wrote:
>> 
>> For #3 there is not much to do actually - just having a means to
>> switch between resources would be sufficient. That means could be a
>> button underneath the video or a second tab with the video and lets
>> the user change between the main video and it's auditory-only
>> counterpart. I wonder if we even need special accessibility features
>> for this.
>> 
>> The idea we have long talked about to use media queries on a <source>
>> element to label the accessibility features of its resource could be very
>> useful here. It would allow an author to include videos with and without
>> open audio descriptions in the markup:
>> 
>>    <video controls>
>>        <source src="trailer_with_open_captions.m4v"
>> media="accessibility(audiodescription:yes)" >
>>        <source src="trailer.m4v">
>>    </video>
>> 
>> and the user agent will automatically choose the captioned file if
>> the user's preferences say they want them.
>> eric
>> 
>> 
> 
> So, the other way of doing this is through introducing a @kind
> attribute like we have it on <track>. We could have
> 
> <video controls>
>       <source src="trailer_with_open_captions.m4v" kind="descriptions" >
>       <source src="trailer.m4v">
>  </video>
> 
> That would similarly signify what type of tracks are available in
> content. It would also probably mean to extend the @kind attribute to
> allowing more than one value, since there could be multiple tracks in
> it.
> 
> I am, however, wondering how either of these solutions will interact
> with the resource selection algorithm. It seems to me that the
> algorithm currently states that when the first resource is found that
> the browser is able to decode, it will stop and not search for more.
> So, even if we say that a user would prefer to listen to a video with
> audio descriptions over a video without audio descriptions, we cannot
> make the lack of the @kind attribute or the @media attribute one that
> completely dismisses a <source> element. If, for example in the above
> example the <source> elements are ordered the other way round like
> this:
> 
> <video controls>
>      <source src="trailer.m4v">
>     <source src="trailer_with_open_captions.m4v"
> media="accessibility(audiodescription:yes)" >
> </video>
> 
> then the browser would always pick the one without the audio
> description first, since it cannot assume that one with audio
> description will eventually come.
> 
> Do you have a solution for this case?
> 
> Silvia.
>

Received on Friday, 3 December 2010 18:02:08 UTC