Re: TextTrackCue discussions

On Thu, Sep 19, 2013 at 9:21 PM, Silvia Pfeiffer
<silviapfeiffer1@gmail.com>wrote:

> On Thu, Sep 19, 2013 at 12:11 AM, Philip Jägenstedt <philipj@opera.com>
> wrote:
> > On Wed, Sep 18, 2013 at 4:05 PM, Glenn Adams <glenn@skynav.com> wrote:
> >>
> >> On Wed, Sep 18, 2013 at 2:42 AM, Philip Jägenstedt <philipj@opera.com>
> >> wrote:
> >>>
> >>> On Wed, Sep 18, 2013 at 9:59 AM, Silvia Pfeiffer
> >>> <silviapfeiffer1@gmail.com> wrote:
> >>> > On Tue, Sep 17, 2013 at 10:15 PM, Philip Jägenstedt <
> philipj@opera.com>
> >>> > wrote:
> >>> >> On Tue, Sep 17, 2013 at 2:11 PM, Silvia Pfeiffer
> >>> >> <silviapfeiffer1@gmail.com> wrote:
> >>> >>> On Tue, Sep 17, 2013 at 6:01 PM, Philip Jägenstedt <
> philipj@opera.com>
> >>> >>> wrote:
> >>> >>>> On Tue, Sep 17, 2013 at 12:52 AM, Silvia Pfeiffer
> >>> >>>> <silviapfeiffer1@gmail.com> wrote:
> >>> >>>>
> >>> >>>>> UnparsedCue can be created by JS and the JS could try to add it
> to a
> >>> >>>>> track that has a @kind=captions.
> >>> >>>>
> >>> >>>> Why do we need to allow scripts to create UnparsedCue at all? That
> >>> >>>> will require some new API surface that isn't needed to solve the
> >>> >>>> original use case -- in-band metadata tracks which aren't just a
> >>> >>>> special-case of a captioning format.
> >>> >>>
> >>> >>> It also allows JS devs to create @kind=metadata cues without
> having to
> >>> >>> decide to use a more specific format such as WebVTT.
> >>> >>
> >>> >> Given that UnparsedCue would be a strict subset of VTTCue, that
> >>> >> doesn't sound at all worth adding APIs for.
> >>> >
> >>> > It avoids developer confusion, which is sufficient reason for me.
> >>> >
> >>> > Here's another use case: when a browser exposes in-band text tracks
> >>> > through a @kind=metadata TextTrack, this allows developers to make
> >>> > corrections to the list of cues - add cues if necessary to e.g. fill
> >>> > gaps.
> >>>
> >>> It turns out I'm not up to date with what the spec says. addCue and
> >>> removeCue are now on the TextTrack interface and take a TextTrackCue,
> >>> whereas there used to be a function which created *and* added a cue. I
> >>> thought we'd need a new such function, but it's only a question of
> >>> whether or not UnparsedCue has a constructor.
> >>>
> >>> I would prefer if UnparsedCue was only ever used for in-band metadata
> >>> tracks of an unknown kind, but if browsers are going to expose the
> >>> UnparsedCue interface anyway, then exposing a constructor by that same
> >>> makes no difference, and would indeed be more consistent.
> >>
> >>
> >> Would it be better to call this interface RawCue or RawTextCue, so that
> it
> >> doesn't necessarily imply that it is not parsed? This would be useful to
> >> permit a rendered cue interface type (that exposes getCueAsHTML) to
> inherit
> >> from Raw{Text}Cue in order to provide the text attribute.
> >>
> >> I have specified a RawTTMLCue to serve this function (as a base
> interface
> >> for TTMLCue) in [1].
> >
> > Hmm, this makes we wonder what the property for accessing the data is
> > supposed to be. If it's a string, then it requires that the UA at
> > least know what the encoding of the text is, which seems like it might
> > not be true. RawCue and exposing the data as a typed array .data
> > property seems OK to me. The other option is to base64-encode cues
> > which are of an unknown encoding, I guess?
>
> ArrayBuffer .data works for me.
> Should we keep .text as well, because converting from ArrayBuffer to
> String can be inefficient, see
>
> http://updates.html5rocks.com/2012/06/How-to-convert-ArrayBuffer-to-and-from-String
> ?
>
> Also, I am not fussed about the naming - RawCue works for me just like
> UnparsedCue.
>
> However, I would suggest not to inherit TTMLCue from RawCue, since as
> Simon pointed out earlier, you'd like JS devs to be able to do
>
>    var raw_cue = cue instanceof RawCue;
>
> to detect that a cue is not taken care of by the UA. If TTMLCue
> inherits from it, that doesn't work.
>

But cue instanceof RawCue && ! cue instanceof TTMLCue would work. However,
I don't particularly like tying the cue type to assumptions about UA based
rendering support. In generally, it should be possible for an application
to perform its own rendering independently of whether the UA is capable of
rendering a track's cues (or a specific cue). I feel we haven't quite
resolved the matter of whether @kind captions/subtitles are always (and
only) rendered by UA and @kind metadata are never rendered by UA. While I
concur that it doesn't make sense for a UA to render @kind metadata, just
because a UA can render @kind captions/subtitles doesn't logically imply
that it should do so, particularly if application logic wants to perform
its own rendering or modify cue content prior to UA rendering.