Re: [media] Moving forward with captions / subtitles

Hi John,

On Fri, Feb 19, 2010 at 1:39 PM, John Foliot <jfoliot@stanford.edu> wrote:
>
> I am increasingly becoming worried about the reliance of client side
> scripting in these messages/suggestions/proposals (as I am reading things
> - correct me if I am wrong).  Regulatory compliance here is quite clear;
> today critical functionality must be maintained when client side scripting
> is not supported (WCAG1, Section 508). If the API enhances or improves the
> ability to do key functional things, or allows for alternate behaviors and
> 'style' considerations great, but anything beyond that is going to receive
> a fair bit of pushback from certain quarters.

You shouldn't be worried about it. Once everything is implemented, all
the tracks available inside a media resource and the tracks added
through markup are meant to be exposed through default display and
controllable through default menus.

There is, however, the need for those corporates that want to roll
their own user interface controls for media resource (such as almost
any company that wants to put their own styling and logo onto player
controls) to have an alternative means of controlling the user
interface. For this, there is a requirement to expose everything that
the UI does by default also in a JavaScript API. That's what this is
for.

I actually thought Eric explained that well in the media subgroup meeting.


> OK, so I am a little (lot?) confused. In an off list email I asked Silvia
> a question about <tracks> inside and outside of the media asset, and about
> order of precedence. The above confuses me even more: is @enabled an
> attribute for tracks both inside *and* outside of the media element?

@enabled is an attribute in the markup of outside media elements (see
http://www.w3.org/WAI/PF/HTML/wiki/Media_TextAssociations).

enabled is also a boolean field in the JavaScript API for inside media
elements (see http://www.w3.org/WAI/PF/HTML/wiki/Media_MultitrackAPI).

What we have done recently is to say that the JavaScript API the works
for inside media elements is also usable for outside media elements,
since we have designed them to be virtually identical. This is nice,
because now a JavaScript user doesn't have to worry about where its
accessibility tracks come from - they can address them all in the same
way.


> When
> I asked about precedence, the answer I received was that all tracks
> ultimately would be exposed in the UI - both those inside (extracted by
> the JS API) and 'hard coded' into the page.

(I will assume that by 'hard coded' you mean "declarative markup" on the page)

Those inside are not just exposed through JavaScript. I think this is
where the confusion originates. They are also exposed through the user
interface in a menu. You have to think about this as similar to how
the pause, play, and transport bar controls are exposed through a user
interface, but can at the same time be controlled through JavaScript.
There is no declarative markup for the controls necessary since anyone
wanting to use video will want to activate those controls. The same
here: there is no declarative markup of the tracks inside a media
resource necessary since they will be part of the controls.


>  In [1] above, which is
> 'first'? If I have a track burned into the media that is 'enabled', and
> another track that is hard-coded in the web page that is 'enabled' - well,
> you can't have 2 can you?

If you as the Webpage author enables both, then yes, you will have both enabled.


> In [2] it seems a little
> clearer to me, as the user should ultimately decide - but here I raised a
> second question: what if there are 2 very similar but yet different
> 'tracks' that meet those basic criteria "caption/EN", where the first one
> is 'burned in', and the second, slightly modified version is referenced
> via hard-coding.  They are sufficently different, yet tagged the same. Now
> what?

'Burnt in' captions are not controllable. They are part of the pixels
of the video just like any other text that displays in the video. If
you enable captions on top of a video that has burnt in captions, you
will see the text captions displayed over the top of the video (and
thus potentially over the top of the burnt in captions).

Assuming you are also referring to DVD style captions, which are
actually images that are overlayed on top of the video. In this case,
the browser would probably enable that track because it's marked as
captions in English, and would probably also enable the external text
track which has captions in English. The user would notice, go into
the menu and turn one off manually. Or the Web page author would
notice (which you would hope) and turn one off in JavaScript, so it
would not reach the user. But ultimately, the user can go into the
menu and make their own selection which overrules everything the
browser or the Web author have preset for them.


> And in [3]... well back to my concern about client side scripting...
> and what if the original source file has "captions/EN" enabled, but the
> page author specifies "captions/SP" enabled...?

I am not sure there if MP4 allows to set a track as enabled (or
provides such a hint to the media players). Ogg certainly doesn't. So
there is no such thing as "the original source file has captions/EN
enabled. However, the browser may have a setting to auto-enable
captions in English, so, indeed, it is possible that the page author
thinks you should always see the captions in SP (whatever language
that is), while you prefer to see them in English and then you end up
seeing them both. That is bad authoring in my opinion, unless the page
is written in SP and the author only expects people that read SP to
read his page and watch the video. In which case it's probably ok to
expect anyone else who speaks another language to go in and manually
turn of the SP subtitles. Ultimately it always comes back to user
control.


> I've been encouraged to ask these questions more openly, so, am I missing
> something fundamental here?

I am glad you're doing so! I'm sure there are many people that didn't
follow all the discussions in detail and ask similar questions. Now
they can either read up on them here, or ask you or anyone else who
has read this email to explain it to them.

I hope I was able to explain.

Cheers,
Silvia.

Received on Friday, 19 February 2010 03:14:25 UTC