Re: Resolving TextTrackCue issues

On Fri, Sep 6, 2013 at 11:24 PM, Silvia Pfeiffer
<silviapfeiffer1@gmail.com>wrote:

> On Fri, Sep 6, 2013 at 2:43 AM, Glenn Adams <glenn@skynav.com> wrote:
> >
> >
> > On Thu, Sep 5, 2013 at 8:19 AM, Silvia Pfeiffer <
> silviapfeiffer1@gmail.com>
> > wrote:
> >>
> >> > Could we compared example codes?
> >>
> >> I can give you an example: if you have TTML in-band in MP4, it's
> >> caption content, a browser has no parser and renderer for it, but can
> >> in theory extract the cues from the MP4 encapsulation -
> >>
> >> - the WHATWG spec would either not expose them to JS at all, or expect
> >> them to be exposed as VTTCue objects with @kind=metadata
> >
> >
> > This would not work, since VTTCue interprets cues of kind metadata as
> WebVTT
> > metadata text [1], which is most definitely incompatible with TTML that
> has
> > been serialized into intermediate synchronic document instances, each of
> > which is effectively an XML document.
> >
> > [1]
> >
> https://dvcs.w3.org/hg/ttml/raw-file/tip/ttml2/spec/ttml2.html#extension-designations
>
> So this is what we have to resolve. Why do you think VTTCue of
> @kind=metadata can't contain TTML intermediate synchronic document
> instances (i.e. a XML document) ?
>

1. Because WebVTT metadata text is incompatible with representing a
serialized TTML document entity unless one applies a transfer encoding
first, such as BASE64.
2. Because TTML is not related to VTT, and pretending that TTML is WebVTT
metadata text just  confuses authors and implementers.
3. Because TTML content will not be @kind=metadata, but rather captions,
subtitles, descriptions, etc. In other words, a renderable @kind.

If a browser wants to expose TTML content to client script and it doesn't
support rendering of TTML (either directly or by translating internally
into VTT), then it should expose unparsed TTML to client script using an
UnparsedCue or GenericCue interface's text attribute.

So let's resolve the matter by defining an UnparsedCue (or whatever folks
would like to name it).

Absent doing this for general use in the HTML spec, I expect that TTML2
will define an UnparsedTTMLCue interface as a sub-interface of TextTrackCue
as currently defined in the WHATWG spec. I expect this will result in the
following:

interface TextTrackCue {
  ...
}

interface UnparsedTTMLCue : TextTrackCue {
  readonly attribute DOMString text; // TTML ISD as XML entity serialized
to UTF-16, i.e., an ES String
}

[Constructor(optional DocumentFragment ttml)]
interface TTMLCue : UnparsedTTMLCue {
  DocumentFragment getCueAsTTML(); // TTML ISD as XML
  DocumentFragment getCueAsHTML(); // rendered TTML ISD as HTML
}

I haven't decided if the TTML and/or HTML fragments should be live or not.
Possibly for the TTML fragment, probably not for the HTML rendering.

Along with these interface definitions, I expect there will be normative
text that requires a presentation processor to expose UnparsedTTMLCue
instances in the case that the UA does not parse or render TTML, and, if it
does render TTML, then it must expose TTMLCue instances containing
populated TTML and HTML fragments.