Re: [media] Moving forward with captions / subtitles

Hi Eric,

On Sun, Feb 14, 2010 at 3:42 AM, Eric Carlson <eric.carlson@apple.com> wrote:
>
>> On Feb 13, 2010, at 7:01 AM, Philip Jägenstedt wrote:
>>
>>> On Sat, 13 Feb 2010 21:04:36 +0800, Silvia Pfeiffer
>>> <silviapfeiffer1@gmail.com> wrote:
>
>
>> I think it would be good to treat them as the same as far as possible,
>> including in the DOM API and MediaTracks collection. That way the same user
>> JavaScript could operate on the resource without caring if the tracks are
>> resource-internal or added using <track>.
>
>   I agree that external track references should be treated the same as
> tracks in the media file. There is no distinction in the final presentation
> to the user, so I think they should also be treated the same by the UA when
> choosing among alternates and when presenting the DOM API.

I agree. I've already applied the name change. :-)

>   I like the idea of reusing <source> to list alternate tracks. We should
> also include the "media" attribute, it would help in defining the selection
> criteria. Can we use "media" instead of adding a "lang" attribute?

The spec already has the "media" attribute. As for "lang" - I'm happy
to put it into "media", but it might be useful as a general attribute
for audio and video, too.


>>> However, I must say I really like the idea of making it independent of
>>> "text", i.e. leaving the possibility open to add "tracks" of audio or
>>> video in future.
>>>
>>> I'd be happy for something that essentially means "external parallel track".
>>
>> Considering how many different names we have already come up with, I doubt
>> <track> will be the last :) Brainstorm away!
>
>   To me the singular "track" implies that only one <source> will be chosen
> which will not be true in all cases. Maybe <tracks>?

I actually think that within a list of tracks only one should ever be
activated. I don't think it makes sense to have more than one subtitle
track active at a time. For those who have that need, they can always
do that in JavaScript. It certainly doesn't make sense to have more
than one caption track active, or more than one textual audio
description, or more than one lyrics or karaoke track.



>>>> role="" is fine, but I'd like to see more ideas on what UAs should to with
>>>> it.
>>
>> Yes, but is it expected that the UA should do something with the attribute,
>> like make context menus based on it? Or should it be part of the track
>> selection algorithm? (Where "track selection algorithm" does not exist yet,
>> but is what will select which tracks are enabled by default based on...
>> language and such?)
>
>   I think the selection of alternates is an important point. Some media
> container formats (eg. QuickTime and MPEG-4)  allow an author to mark tracks
> as begin part of an "alternate group". This instructs the media engine to
> enable only one track in the group based on a condition on the user's
> machine when the file is opened for playback. For example, a movie can
> have subtitle tracks and chapter tracks in multiple languages, but only one
> of each is rendered when the movie plays.
>   We need to support this use case with external "tracks", and we need to
> define the selection algorithm when a file has both internal and external
> tracks.

I agree with this.


>   We also need to define a mechanism to mark tracks as being part of an
> alternate group. Is an attribute on <source> enough?
>     <tracks>
>         <source type="text/srt" src="en-captions.srt" lang="en"
> role="caption">
>         <source type="text/srt" src="zh-captions.srt" lang="zh"
> role="caption">
>
>         <source type="text/srt" src="en-chapters.srt" lang="en"
> role="chapters">
>         <source type="text/srt" src="zh-chapters.srt" lang="zh"
> role="chapters">
>     </tracks>
>
> Or should we have a grouping element like Silvia had in her early proposal?
>     <tracks>
>         <track role="caption">
>             <source type="text/srt" src="en-captions.srt" lang="en">
>             <source type="text/srt" src="zh-captions.srt" lang="zh">
>         </track>
>
>         <track role="chapters">
>             <source type="text/srt" src="en-chapters.srt" lang="en">
>             <source type="text/srt" src="zh-chapters.srt" lang="zh">
>         </track>
>     </tracks>
>   I hesitate to define yet another element, but I think the markup in a
> complex case like Silvia's Elephants Dream
> sample, http://annodex.net/~silvia/itext/elephant_no_skin_v2.html, is
> clearer because of it. On the other hand, will complex cases like this be
> common enough that we need it?

The two-level mechanism is already in the current proposal. I think it
is more flexible. But we can always do it the way in which the video
and audio element work and directly attach a src to the track if there
is only one:

<video src="video.ogg">
  <track src="captions.srt" type="text/srt">
</video>

This makes it really flexible and simple. However, it might encourage
ppl to do stupid things such as list all their alternative subtitle
tracks as parallel tracks. OTOH, that could be a way to do more than
one parallel track. These would probably not have a role, since there
is no distinction between these tracks.

Cheers,
Silvia.

Received on Sunday, 14 February 2010 01:04:42 UTC