Re: TextTrack API changes

From: Silvia Pfeiffer <silviapfeiffer1@gmail.com<mailto:silviapfeiffer1@gmail.com>>
Date: Tuesday, April 23, 2013 3:21 PM
To: Glenn Adams <glenn@skynav.com<mailto:glenn@skynav.com>>
Cc: Bob Lund <b.lund@cablelabs.com<mailto:b.lund@cablelabs.com>>, "Jerry Smith (WINDOWS)" <jdsmith@microsoft.com<mailto:jdsmith@microsoft.com>>, Mark Vickers <mark_vickers@cable.comcast.com<mailto:mark_vickers@cable.comcast.com>>, public-html <public-html@w3.org<mailto:public-html@w3.org>>
Subject: Re: TextTrack API changes

To compare the two new interfaces look at:
http://dev.w3.org/html5/webvtt/#webvtt-api (for WebVTTCue)
and
http://www.w3.org/html/wg/drafts/html/master/single-page.html#texttrackcue (for TextTrackCue)
.

I had the discussion about .text and getCueAsHTML() with Ian, too. Here are his thoughts on why not to add .text to the attribute list of TextTrackCue:


> .text provides the uninterpreted content of a cue - every format will
> have such.

Not necessarily. DVD subtitles wouldn't have one, they'd likely have
something that returned an ImageBitmap or some such. TTML probably
wouldn't have one, since their format is structured DOM and might never
have had a textual form (think script-created DOMs) -- they'd just have a
way to get a pointer to the relevant node in the actual DOM tree, I'd
guess. Other formats might have other needs, e.g. EBU might want to just
expose the Text Field as a raw 112 byte ArrayBuffer; I don't know how
you'd really expose an unparsed DOMString for EBU.

The above cases argue for track format-specific APIs for exposing Cues of these formats in a form other than text or doc fragement. They also argue that some text track formats would not have a valid text or HTML return. However, they don't really address whether most formats have a valid form returned by …Text and …HTML.


At this point, it actually doesn't matter how we make the distinction between what attributes / functions stay on TextTrackCue and which go to WebVTTCue, because the only one that can be instantiated is WebVTTCue. When TTMLCue or some other format comes along and creates another interface, and we find out that there are more shared attributes, then we can still make that change. It's probably better to stay minimal until then.

We've already implemented in-band captions for MPEG-2 TS where getCueAsHTML returns a valid doc fragment for Cues containing 708 captions. We've also implemented a UA that exposes all other MPEG-2 TS  text track data as Base64 encoded text via getCueAsText for text tracks not recognized by the UA. This is to provide Web applications with the opportunity to handle text track formats not supported by the UA. There is at least one 3rd party Web app developer that wants to make use of this feature.



Regards,
Silvia.

On Wed, Apr 24, 2013 at 4:53 AM, Glenn Adams <glenn@skynav.com<mailto:glenn@skynav.com>> wrote:
I agree with Bob. I'm afraid I didn't look at the API details off the changes, but I'd suggest that Sylvia summarize which API features she would like to move from TextTrackCue to WebVTTCue, and that we can then review which of those features should remain generic.


On Tue, Apr 23, 2013 at 12:37 PM, Bob Lund <B.Lund@cablelabs.com<mailto:B.Lund@cablelabs.com>> wrote:
It seems to me that the distinction between "replaced" vs "reorganized" has to do with what got moved from TextTrackCue to WebVTTCue. I would have thought that getCueAsHTML and getCueAsText would be generic across formats and therefore a candidate to remain with TextTrackCue.

Bob

From: Silvia Pfeiffer <silviapfeiffer1@gmail.com<mailto:silviapfeiffer1@gmail.com>>
Date: Tuesday, April 23, 2013 2:57 AM
To: "Jerry Smith (WINDOWS)" <jdsmith@microsoft.com<mailto:jdsmith@microsoft.com>>
Cc: Glenn Adams <glenn@skynav.com<mailto:glenn@skynav.com>>, Bob Lund <b.lund@cablelabs.com<mailto:b.lund@cablelabs.com>>, Mark Vickers <mark_vickers@cable.comcast.com<mailto:mark_vickers@cable.comcast.com>>, public-html <public-html@w3.org<mailto:public-html@w3.org>>

Subject: Re: TextTrack API changes

To add to that, we've not *replaced* TextTrackCue with WebVTTCue, but we have reorganised TextTrackCue as an abstract cue interface and created WebVTTCue only in the WebVTT spec. Note also that <track> continues to exist as is and will continue to take WebVTT in its @src attribute (as well as TTML in IE10).

I hope that addresses your concerns?

Thanks,
Silvia.

On Tue, Apr 23, 2013 at 10:51 AM, Glenn Adams <glenn@skynav.com<mailto:glenn@skynav.com>> wrote:
There is no part of this change that entails creating a new element (tag) for different tag formats. Rather, this change improves the definition of some of the text track API interface to move VTT specifics out of the HTML5 spec. This is entirely appropriate since it is expected that TTML (and other formats) will be used for time text track content. In fact I believe IE supports TTML to some extent (though I'm not familiar with the details of this support).

On Mon, Apr 22, 2013 at 12:26 PM, Jerry Smith (WINDOWS) <jdsmith@microsoft.com<mailto:jdsmith@microsoft.com>> wrote:
I have some concerns about these changes.  They create a new element that is specific to a file format.  Format specifics like this are normally abstracted away.  For instance, for images we don’t have <jpegimage>, <pngimage> etc…  It would be very inconsistent to have WebVTT variants for TextTrack.

What are the plans for other captioning formats?  Would we similarly propose having a ttmptextcue object?

Jerry

From: Silvia Pfeiffer [mailto:silviapfeiffer1@gmail.com<mailto:silviapfeiffer1@gmail.com>]
Sent: Wednesday, April 17, 2013 5:26 PM
To: Bob Lund
Cc: Mark Vickers @ Comcast; Glenn Adams; public-html

Subject: Re: TextTrack API changes

I will apply this to HTML5.0 next week if there are no objections.
Cheers,
Silvia.

On Thu, Apr 18, 2013 at 3:28 AM, Bob Lund <B.Lund@cablelabs.com<mailto:B.Lund@cablelabs.com>> wrote:
+1

From: <Vickers>, Mark Vickers <mark_vickers@cable.comcast.com<mailto:mark_vickers@cable.comcast.com>>
Date: Wednesday, April 17, 2013 11:21 AM
To: Glenn Adams <glenn@skynav.com<mailto:glenn@skynav.com>>
Cc: Silvia Pfeiffer <silviapfeiffer1@gmail.com<mailto:silviapfeiffer1@gmail.com>>, public-html <public-html@w3.org<mailto:public-html@w3.org>>
Subject: Re: TextTrack API changes
Resent-From: <public-html@w3.org<mailto:public-html@w3.org>>
Resent-Date: Wednesday, April 17, 2013 11:22 AM

I'd very much support this change as it will significantly improve TextTrack. Though, I think it should be made to both 5.0 & 5.1 or neither, to avoid backwards-incompatibility.

Thanks,
mav

On Apr 8, 2013, at 2:26 PM, Glenn Adams <glenn@skynav.com<mailto:glenn@skynav.com>> wrote:



On Sun, Apr 7, 2013 at 11:42 PM, Silvia Pfeiffer <silviapfeiffer1@gmail.com<mailto:silviapfeiffer1@gmail.com>> wrote:
Hi all,
Recently, I cherry-picked some changes to the TextTrack API from the WHATWG repository into the HTML5 specification.

In particular, I am referring to these patches:

** Split TextTrackCue into an abstract TextTrackCue interface and a WebVTT-specific interface WebVTTCue. Makes it easier to use TextTrack with other file formats.
https://github.com/w3c/html/commit/586ae3996fdce5d9f71cbe57a08759fce7b26d8f
WHATWG: 98cdbf20015b11ae7febc581280c3ce02dcd800e (7742)

** Split more WebVTT-specific things into the WebVTT spec. This also makes some normative changes to HTML for handling non-WebVTT cue types, but that shouldn't affect any existing implementations.
https://github.com/w3c/html/bdae138d123ddb73586eb8d7f39761ec93e3aa28
WHATWG: 0776094323b3f44cbf88eb9f023f4b12c3a6b6a9 (7748)

The aim of these patches was two-fold:

Firstly, they provide for a cleaner cut between the WebVTT specification and the HTML specification. This was in preparation for a removal of the WebVTT text from the source file from which the HTML specification is created such that the WebVTT specification can now be edited separately (see https://dvcs.w3.org/hg/text-tracks/raw-file/default/webvtt/webvtt.html).

Secondly, these changes make the TextTrack API abstract and thus more easily extensible to other file formats such as TTML.

The downside of the changes is that TextTrackCue is now an abstract interface without a constructor (instead, the WebVTT spec provides the WebVTTCue constructor). This breaks existing implementations of the TextTrackCue interface in webkit-based browsers (including blink) and in presto. IIUC, Mozilla and IE are not supporting TextTrackCue yet. Also, analysis on the webdevdata collection suggests that the TextTrackCue constructor is not used much on the Web yet, so this is still a good time to break the interface (see http://lists.w3.org/Archives/Public/public-texttracks/2013Apr/0006.html).

While I have right now only applied these changes to HTML5.1, I am considering applying them to HTML5.0 as well if presto, webkit and blink decide to change their implementation and gecko and trident decide to support the new specification. I am looking for advice on such a move.

Thanks for doing this. I think this makes this functionality more useful and more consistent with existing MIME type independent interfaces. Cox supports these changes.