[Bug 21851] revert removal of text and getCueAsHTML members; address constructor changes

https://www.w3.org/Bugs/Public/show_bug.cgi?id=21851

--- Comment #16 from Silvia Pfeiffer <silviapfeiffer1@gmail.com> ---
(In reply to comment #15)
> 
> In my mental model, I don't (thus far) distinguish between the content type
> of a cue and the content type of a text track.

It is important to do so, because it implements what you called the "fallback
order". A browser may be able to parse a text track into its individual cues.
But it may not know what to do with the cues - in fact, for track of
kind=metadata, that is exactly the idea.


> Rather, I view the content of
> a cue to be a derivation|compilation of some portion of the content of a
> text track, where the definition of that derivation is intrinsically tied to
> the definition of the content type of the track.

Thus you merge the content type of the track and the cue into one. That won't
work for WebVTT, which allows to deliver cues for different kinds that need
different parsers. I don't think you even want that for TTML, seeing how I
recently saw an example of a TTML file that delivers kind=metadata tracks
containing JSON.


> More specifically, when I use the term content type I'm referring to a
> something that could or does have a IANA Media Type assigned to it.

Knowledge of a mime type of the text track is of no consequence to the JS
developer. It is abstracted away by the browser by providing a TextTrack object
with a list of cues. The only format that is of any relevance to the JS
developer is the format of the cues.


> While
> it's possible to treat TextTrackCue.text as a text serialization of some
> content type independently of the content type of the originating track, I'm
> not sure of the utility to that, unless there are multiple possible
> derivations that could end up producing different sub-formats for
> TextTrackCue.text from a given originating track's content type.

See my above example of a WebVTT (or a TTML) file that could contain captions
to be interpreted as HTML or as JSON.

> Even in
> such a case, I would expect that other attributes of the cue or the actual
> content of TextTrackCue.text could be used to distinguish among these
> sub-formats.

Other than educated guessing, the JS has nothing to go on.


> > That is indeed a good question, but one to be discussed in the CG. I
> > personally think that we need to distinguish between the encapsulation file
> > type and the cue file type and that cue file types can have different
> > interpretation algorithms even for the same encapsulation file type.
> 
> By "encapsulation file type" do you mean a container stream like MPEG-2 TS
> or MPEG-4 Part 12 (ISO Base Media File Format) or MPEG-4 Part 14 (MP4 File
> Format)?

Yes.

> Or do you mean a text track format that can encapsulate different
> sub-types? I'd guess you are referring to the ability of text/vtt to carry
> different "kind"s of text: captions, metadata, etc., yes?

"sub-types" are the cue types.


> Do you expect to register (or be able to register) IANA Media Types for
> these sub-formats?

No. They are cue formats and not mime types (file types). You wouldn't register
a mime type on a TTML cue object [1] either. However, your TTML cue object
could contain cue text to be interpreted for a caption or subtitle track, or it
could be interpreted as JSON for a metadata track.

[1] http://www.cwmwenallt.com/ttml/TTMLmapping.htm#ttml-cue


> If not, then it might be better to use a different term
> to describe the format of TextTrackCue.text. Perhaps
> TextTrackCue.textFormat?

Since the cues of a TextTrack all have to have the same cue format, I suggested
adding it to the TextTrack as .cueType. 


> Where the (what you call) encapsulating content
> type (i.e., IANA Media Type of the containing text track resource)
> determines the acceptable values of textFormat?

The encapsulating content can only provide part of the information, just like
the inBandMetadataTrackDispatchType consist of more than just the file format.

For example, for WebVTT cues that come from a WebM file it ends up being a text
track CodecID of “D_WEBVTT/kind“, where kind is one of SUBTITLES, CAPTIONS,
DESCRIPTIONS, or METADATA. The "kind" in this case doesn't actually have a
semantic meaning, but is more like a mapping to the interpretation algorithm to
be used on the cues (i.e. it's a codecID).

[2]
http://www.w3.org/html/wg/drafts/html/master/single-page.html#steps-to-expose-a-media-resource-specific-text-track

To explain the point even better, for WebVTT, I'd eventually like to see
kind=descriptions being interpreted through the speech synthesis API [3], which
can take both pain text and SSML. So, SSML needs a specific interface. I'd
prefer this to not be called WebVTTSSMLCue, but rather something generic like
SSMLCue.

[3] https://dvcs.w3.org/hg/speech-api/raw-file/tip/speechapi.html



> > I think we could change that, while there is no implementation of that
> > attribute in browsers yet. We would lift both restrictions and could rename
> > it to @cueType and it would provide a hint as to what algorithm to use to
> > interpret the cue content of a TextTrack. E.g. "D_WEBVTT/captions" would use
> > the caption/subtitle rendering algorithm of WebVTT, while
> > "D_WEBVTT/chapters" would use a different rendering algorithm and
> > "D_WEBVTT/metadata" would not do any rendering interpretation at all. Other
> > formats would set defined types, too.
> 
> OK, this sounds like what I meant by textFormat above, a text track media
> type specific enumeration of possible formats to be used for the purpose of
> interpreting TextTrackCue.text, yes?

Yes.


> > > The main problems are not technical, they are (1) spec uncertainty
> > > (introducing possible backwards incompatible changes for 5.0 to 5.1), (2)
> > > publishing schedule, and (3) ipr commitments.
> > 
> > The idea is to introduce the changes here to 5.0, too, if all browsers agree
> > to implement them.
> 
> OK, but "all" browsers may be too zealous. In any case, if at least two of
> the set of {IE, MOZ, WK} are amenable to changes for 5.0 in this regard,
> then that should be sufficient IMO.

Sure. :-)

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

Received on Monday, 13 May 2013 03:37:00 UTC