- From: Bob Lund <B.Lund@CableLabs.com>
- Date: Thu, 5 Sep 2013 19:22:20 +0000
- To: Silvia Pfeiffer <silviapfeiffer1@gmail.com>
- CC: Eric Carlson <eric.carlson@apple.com>, Jer Noble <jer.noble@apple.com>, Glenn Adams <glenn@skynav.com>, Philip Jägenstedt <philipj@opera.com>, public-html <public-html@w3.org>, Ian Hickson <ian@hixie.ch>
On 9/5/13 8:22 AM, "Silvia Pfeiffer" <silviapfeiffer1@gmail.com> wrote: >On Thu, Sep 5, 2013 at 1:52 AM, Bob Lund <B.Lund@cablelabs.com> wrote: >> >> >> On 9/4/13 12:48 AM, "Silvia Pfeiffer" <silviapfeiffer1@gmail.com> wrote: >> >>>On Wed, Sep 4, 2013 at 2:14 PM, Eric Carlson <eric.carlson@apple.com> >>>wrote: >>>> >>>> On Sep 3, 2013, at 5:16 PM, Silvia Pfeiffer >>>><silviapfeiffer1@gmail.com> >>>>wrote: >>>> >>>>> On Wed, Sep 4, 2013 at 9:19 AM, Eric Carlson <eric.carlson@apple.com> >>>>>wrote: >>>>>> >>>>>>> Do you expose the existance of these in-band captions somehow to >>>>>>>JS? >>>>>>> >>>>>>> I'm concerned that if the browser renders captions automatically on >>>>>>> top of video without the JS developer being able to find out about >>>>>>>it, >>>>>>> how would the JS developer know that there are captions and to >>>>>>>avoid >>>>>>> rendering another lot themselves - or rendering something else in >>>>>>>the >>>>>>> space of the captions? >>>>>> >>>>>> On versions of the OS where it is possible for WebKit to "take >>>>>>over" >>>>>> rendering of in-band captions from the media engine, they behave >>>>>>just >>>>>>like >>>>>> out-of-band tracks: in-band tracks are part of the video.textTracks >>>>>>and the >>>>>> cues are part of track.cues/activeCues (when appropriate). >>>>> >>>>> So for the JS dev they are exposed as instances of the old >>>>> TextTrackCue interface? >>>> >>>> Yes. >>>> >>>>> The VTTCue interface sufficiently satisfies this use case then? >>>> >>>> True, but the in-bad cues may or may not be WebVTT originally. Is >>>>there an advantage to using VTTCue instead of the old TextTrackCue >>>>interface? >>> >>>getCueAsHTML() of VTTCue interprets what's in .text as WebVTT content. >>>Exposing other format content in that way would not make much sense, >>>just as parsing HTML through a Word Doc format parser makes not much >>>sense. So, IMHO neither VTTCue nor the old TextTrackCue interface are >>>appropriate here since you're not dealing with WebVTT content >>>originally. >>> >>> >>>>> What is the content in .text ? >>>>> >>>> The cue text. >>> >>>Just to clarify: Is that a plain text version of the original cue >>>content? For example, for CEA608 content from a SCC file it would just >>>be the plain text of the cue stripped of all the other characters? Or >>>in the case of SRT, all tags are stripped and just the plain text is >>>exposed? >>> >>>If you do that, then we're just pretending the in-band text track was >>>actually WebVTT content. >>> >>>If you don't return plain text, but the actual original content of the >>>cue, you get the wrong behaviour with getCueAsHTML() and the rendering >>>algorithm of the old TextTrackCue or the new VTTCue interface. >>> >>> >>>>>> On versions of the OS where the system frameworks do not have the >>>>>> necessary API to override cue rendering, in-band tracks are part of >>>>>> video.tracks so they can be enabled/disabled by script but cues are >>>>>>rendered >>>>>> by the media engine. >>>>> >>>>> In this case, I assume only the existence of the track, but not of >>>>>the >>>>> cues is exposed to JS? I.e. track.cues/activeCues is empty? Or are >>>>>you >>>>> listing fully-abstract TextTrackCue instances here to at least >>>>>provide >>>>> starttime/endtime to the JS devs? >>>>> >>>> Correct, the cues are not available to WebKit at all in this case. >>> >>>OK. Would you consider using the abstract TextTrackCue interface of >>>the WHATWG spec for exposing these cues, so JS developers can at least >>>react to cue change events? >>> >>> >>>>> And an orthogonal question: you've probably seen the Cablelabs spec >>>>> for exposing MPEG-2 in-band text tracks of different types to >>>>>HTML[1]. >>>>> Seeing as both Cablelabs and the HTML spec are trying to accommodate >>>>> using in-band text tracks in HTML/JS, what do you suggest is the best >>>>> way forward to specify this? >>>>> >>>>> * Program Map Table: would you suggest to use VTTCue with >>>>> @kind=metadata, GenericCue, or a new PMTCue interface? >>>>> >>>> It seems to me that text track content that a UA does not render >>>>itself is, by definition, metadata. >>> >>>In this case, that definition works, because no @kind value matches >>>the semantic content of a PMT track. >>>But caption data that is not rendered is still semantically of >>>@kind=captions. >>> >>>> Again, it is not WebVTT data so is there an advantage to VTTCue versus >>>>old TextTrackCue interface? >>> >>>Not between those two, since they are identical. >>> >>> >>>>> * CEA708 track: assuming we don't want to introduce CEA708Cue, how >>>>> would that best be supported? >>>>> >>>> Are any browsers planning to support 708 captions natively? >>> >>>What does "support" mean? >>>Parse them out of a MPEG-2 file (like the Cablelabs spec suggests) and >>>expose them to JS? >>>Or.. Rendering them like you do for formats that the QuickTime >>>framework already supports, but without exposing the original data and >>>pretending it's WebVTT? >>>Or .. go all the way to exposing the format with its own features? >>> >>>Only the 3rd option requires specification of a new interface. The >>>first option requires something like the GenericCue interface, since >>>we need to give the original content of the cue to JS to parse. >>> >>>It seems that Cablelabs expected that there would be browsers that >>>implement parsing, but not rendering for text tracks in MPEG-2. It >>>would be good if there was a statement from browser vendors (or set >>>top box developers that use browser rendering engines or the like) if >>>that is a realistic use case. >> > >> We've implemented UA rendering of 708 captions - this is done by >> converting 708 to WebVTT and then taking advantage of the existing >>WebVTT >> rendering. > >Out of curiosity: In the .text attribute of VTTCue / the old >TextTrackCue interface, are you exposing the 708 cue content or the >converted WebVTT cue content? > > >> The spec calls for making the caption data from the MPEG-2 TS >> available to JS via .text. > >Right. Is that on a track with @kind=captions or @kind=metadata? In the case where the UA can render the CC, @kind=captions, else @kind=metadata > >Thanks for your input! >Silvia.
Received on Thursday, 5 September 2013 19:23:47 UTC