[Bug 21851] revert removal of text and getCueAsHTML members; address constructor changes

https://www.w3.org/Bugs/Public/show_bug.cgi?id=21851

--- Comment #15 from Glenn Adams <glenn@skynav.com> ---
(In reply to comment #14)
> (In reply to comment #12)
> 
> > > I can see a problem with having getCueAsHTML() on the "base class". It
> > > requires application of a conversion algorithm to the original cue content,
> > > converting it to HTML. However, the UA only knows which conversion algorithm
> > > to apply when being told what type the cue is
> > 
> > I do not agree that the UA needs to be told what kind of cue it is. In fact,
> > the UA has better information than the client JS in this regard.
> 
> I can see that being the case for in-band cues, but for cues that are
> created by JS? Hardly.

I'm referring to tracks/cues constructed by the UA, from either in-band or
out-of-band text track content.

> 
> > I agree it
> > would be useful to allow the author to provide authorial hints to the UA in
> > the <track/> element that can help the UA better determine the type of the
> > referenced track resource, but in this case, the authorial hint is akin to
> > specifying @type on <img>.
> 
> I agree that providing information about the file format to the browser is
> useless - if it can interpret it, it will, and if it can't, telling it what
> it is doesn't help.

Actually, it may be important to allow an author to define a fallback order
when multiple resources are available for a track, and those resources employ
different content types. Really, <track> should support <source> just like
video and audio elements. I don't know why it doesn't. If it does, then
providing a content type advisory is useful for the UA (before it attempts to
fetch/decode a resource).

> However, I was referring to the content type of the cue,
> not of the track. If you create a TextTrackCue in JS, you can create it such
> that .text contains a WebVTT caption cue, or a TTML caption cue. Only when
> trying to render it to HTML will you need to interpret it as WebVTTCue or
> TTMLCue.

In my mental model, I don't (thus far) distinguish between the content type of
a cue and the content type of a text track. Rather, I view the content of a cue
to be a derivation|compilation of some portion of the content of a text track,
where the definition of that derivation is intrinsically tied to the definition
of the content type of the track.

More specifically, when I use the term content type I'm referring to a
something that could or does have a IANA Media Type assigned to it. While it's
possible to treat TextTrackCue.text as a text serialization of some content
type independently of the content type of the originating track, I'm not sure
of the utility to that, unless there are multiple possible derivations that
could end up producing different sub-formats for TextTrackCue.text from a given
originating track's content type. Even in such a case, I would expect that
other attributes of the cue or the actual content of TextTrackCue.text could be
used to distinguish among these sub-formats.


> 
> 
> > > For example, assuming that the UA is given a WebVTT file for a <track> of
> > > @kind=metadata whose cues consist of image data URLs. A getCueAsHTML()
> > > function is pretty useless on such a track, and as a JS developer, I would
> > > know that. So I would not want to interpret that track as a WebVTTCue, but
> > > instead just use TextTrackCue.text to access the content, and then interpret
> > > the content as a dataURL that I hand to <img @src> elements.
> > 
> > I guess you are suggesting that the "convert to HTML" algorithm is not well
> > defined for VTT either. Perhaps WebVTTCue shouldn't provide a getCueAsHTML()
> > method either?
> 
> That is indeed a good question, but one to be discussed in the CG. I
> personally think that we need to distinguish between the encapsulation file
> type and the cue file type and that cue file types can have different
> interpretation algorithms even for the same encapsulation file type.

By "encapsulation file type" do you mean a container stream like MPEG-2 TS or
MPEG-4 Part 12 (ISO Base Media File Format) or MPEG-4 Part 14 (MP4 File
Format)? Or do you mean a text track format that can encapsulate different
sub-types? I'd guess you are referring to the ability of text/vtt to carry
different "kind"s of text: captions, metadata, etc., yes?

Do you expect to register (or be able to register) IANA Media Types for these
sub-formats? If not, then it might be better to use a different term to
describe the format of TextTrackCue.text. Perhaps TextTrackCue.textFormat?
Where the (what you call) encapsulating content type (i.e., IANA Media Type of
the containing text track resource) determines the acceptable values of
textFormat?

> 
> 
> 
> > > > It means I can write:
> > > > 
> > > > if (cue.type == "text/x-my-metadata")
> > > >   useTextAccordingToMyMetadataFormat(cue.text);
> > > > else if (cue.type == "text/x-your-metdata")
> > > >   useTextAccordingToYourMetadataFormat(cue.text);
> > > > 
> > > > The demux/decoder can or often will determine the text track format, and can
> > > > provide this as a hint to client JS.
> > > 
> > > I thought that's what the in-band metadata track dispatch type
> > > (http://www.whatwg.org/specs/web-apps/current-work/#text-track-in-band-
> > > metadata-track-dispatch-type) was for (see IDL attribute of TextTrack
> > > http://www.whatwg.org/specs/web-apps/current-work/#texttrack ).
> > 
> > That won't work for two reasons: (1) it is restricted to @kind metadata, and
> > (2) it is restricted to in-band.
> 
> I think we could change that, while there is no implementation of that
> attribute in browsers yet. We would lift both restrictions and could rename
> it to @cueType and it would provide a hint as to what algorithm to use to
> interpret the cue content of a TextTrack. E.g. "D_WEBVTT/captions" would use
> the caption/subtitle rendering algorithm of WebVTT, while
> "D_WEBVTT/chapters" would use a different rendering algorithm and
> "D_WEBVTT/metadata" would not do any rendering interpretation at all. Other
> formats would set defined types, too.

OK, this sounds like what I meant by textFormat above, a text track media type
specific enumeration of possible formats to be used for the purpose of
interpreting TextTrackCue.text, yes?

> 
> 
> 
> > The main problems are not technical, they are (1) spec uncertainty
> > (introducing possible backwards incompatible changes for 5.0 to 5.1), (2)
> > publishing schedule, and (3) ipr commitments.
> 
> The idea is to introduce the changes here to 5.0, too, if all browsers agree
> to implement them.

OK, but "all" browsers may be too zealous. In any case, if at least two of the
set of {IE, MOZ, WK} are amenable to changes for 5.0 in this regard, then that
should be sufficient IMO.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

Received on Sunday, 12 May 2013 13:34:01 UTC