Re: text track associations

Reply inline.


On Sun, Apr 10, 2011 at 8:22 PM, Sean Hayes <Sean.Hayes@microsoft.com> wrote:
> Actually to do it in script, I wouldn't use the mutableTextTrack API,  you can't copy cues from one track to another, you have to recreate them; which assumes the format is the same. I don't think the mutableTextTrack API serves very much purpose actually, if you are having to go to the effort of creating your own text it makes far more sense to re-use the existing dynamic DOM manipulation infrastructure, it's easier and much less restrictive. You would set up a listener on the audio track, use getCueAsHTML() and render into an overlay div. That's what the various polyfill solutions do today.


You get all the different attributes for the mutableTextTrack from the
IDL, so it's not hard to call mutableTextTrack. Using only
getCueAsHTML() works, too, of course, but you still have to do your
positioning yourself and it won't avoid other text on screen, which
the integrated functionality provides.


> It's also not that expensive to create an audio element in script and set it's controller; so since you have to write script, why worry about a markup solution at all?

Indeed, some people may prefer that approach. As I said, for the audio
description example, I would author the captions on the main video as
in this way they will be part of the caption menu on the main video,
too. That won't require any script and just works. You only need a
script solution when you want to do replacement of the captions on the
main video with the ones on the audio element, which is not something
we need for audio descriptions IMO, but would need for a director's
comment.

Cheers,
Silvia.


> -----Original Message-----
> From: Silvia Pfeiffer [mailto:silviapfeiffer1@gmail.com]
> Sent: 10 April 2011 02:28
> To: Sean Hayes
> Cc: David Singer; Mark Watson; Eric Carlson; Ian Hickson; public-html-a11y@w3.org
> Subject: Re: text track associations
>
> Actually, manual rendering of captions from a separate audio track to
> an existing video track isn't that expensive, since it can always use
> the mutableTextTrack API and then it gets the rendering and
> positioning fairly easily. A more markup-related solution could be
> nice though.
>
> Silvia.
>
>
> On Sun, Apr 10, 2011 at 12:41 AM, Sean Hayes <Sean.Hayes@microsoft.com> wrote:
>> If markup for linked audio can't have its captions associated in markup as simply as linking the audio, then the solution is IMO broken and unacceptable. There really aren't many scenarios for linked audio where authors don't need to be able to say, 'these captions go with that audio'. If this isn't built in, it's basically not going to get done; and I think we should actually abandon the idea of linking media resources in markup and have the author do it all in script.
>>
>> -----Original Message-----
>> From: Silvia Pfeiffer [mailto:silviapfeiffer1@gmail.com]
>> Sent: 09 April 2011 11:22
>> To: Sean Hayes
>> Cc: David Singer; Mark Watson; Eric Carlson; Ian Hickson; public-html-a11y@w3.org
>> Subject: Re: text track associations
>>
>> On Sat, Apr 9, 2011 at 5:01 AM, Sean Hayes <Sean.Hayes@microsoft.com> wrote:
>>> Yes they will share a timeline once they are glued together, that's only one of the issues here. They will also need to share a display space. I want to see how this example would be marked up using the 'controller' model. Do I put the director captions inside the <audio> that is the director commentary? If so, how do they show up in the other video space. If not, what information does the UA use to associate the second caption track with the remote audio element?
>>>
>>> For example:
>>> <video id=vid mediagroup=set src=mainvideo.vid >
>>>    <track kind=captions id=main>
>>> </video>
>>>
>>> <audio mediagroup=set src=director.aud >
>>>    <track kind=captions id=director>
>>> </audio>
>>>
>>> how do captions.director end up getting displayed in video.vid?
>>>
>>
>> As it stands right now, in this example, the captions in the audio
>> track are not displayed, so have to be mapped onto the screen through
>> the JavaScript API. So, you can create a display area that matches
>> where you want to place them and you can make it replace the main
>> captions. This is acceptable for complex situation, IMHO.
>>
>> However, we have a similar issue with captions for audio description
>> tracks, which should probably be more commonly displayed as an
>> alternative or addition to the normal captions. I'd probably encourage
>> the Web author to add that in another track with the full transcript
>> of both the captions and audio descriptions as you did in the example
>> below:
>>
>>
>>
>>> Alternately if I do:
>>> <video id=vid mediagroup=set src=mainvideo.vid >
>>>    <track kind=captions id=main>
>>>    <track kind=captions id=director>
>>> </video>
>>>
>>> <audio mediagroup=set src=director.aud >
>>> </audio>
>>>
>>> Then how do I prevent the director captions being offered as an option for the ordinary video until the second audio is selected?
>>
>> In the case of the director's track, I'd probably prefer the above
>> markup. For audio descriptions probably this one and then it's fine
>> when both the captions and audio descriptions are displayed because
>> they would occupy different times anyway.
>>
>>
>> Cheers,
>> Silvia.
>>
>>
>
>

Received on Sunday, 10 April 2011 13:14:00 UTC