[webvtt] Live captioning - JS API review

silviapfeiffer has just created a new issue for 
https://github.com/w3c/webvtt:

== Live captioning - JS API review ==
**To address REQ1, we are after an extension of the JS API.**

The principle idea is that we create a new empty cue with a NULL end 
time, render it, then add text to its content, change its position, 
change text color and other attributes until the cue is complete at 
which stage we know the end time and close it off. To render rollup 
captions, can further create cue regions and successively add new cues
 to them.

The functionality of the 608 live captioning commands will inform what
 needs we have, so here is a first go at a mapping table that we 
created at FOMS. Thanks to 

| Ref | 608 control commands | TextTrackCue API calls
|---|---|--- 
| 1 | start caption text / resume caption text / resume direct 
captioning | `new VTTCue(now(), NULL, '')` - make sure to set the 
defaults as required by 608 - then: `textTrack.addCue(cue)`
| 2 | add a character | `cue.text += char`
| 3 | next row down toggle (includes end all style) | `cue.text += 
'\n'` (may need to end `</c>`,`</i>`,`</b>`,`</u>`)
| 4 | row indicator (one of 15 rows) | `cue.line = row` (whichever row
 calculated)
| 5 | underline toggle | `cue.text += "<u>"` or `cue.text += "</u>"` 
(need to keep track of toggle state)
| 6 | style change (one of 7 text colors and italics) | `cue.text += 
"<c.white>"` (need to have the color style classes pre-defined) and 
`cue.text += "<i>"`
| 7 | 8 ident positions | `cue.positionAlign = offset` (whichever 
offset calculated from ident pos)
| 8 | 8 background colors | `cue.text += "<c.bg_white>"` (need to have
 the background color style classes pre-defined)
| 9 | backspace | `cue.text = cue.text.substr(0, cue.text.length - 1)`
| 10 | delete till end of row | `cue.text = cue.text.substr(0, 
cursor_pos)` (need to keep track of the 608 cursor position)
| 11 | rollup caption with 2, 3 or 4 rows | `new VTTRegion()` then 
`region.lines = x`
| 12 | flash on (srlsy?) | `cue.text += "<c.blink>"`
| 13 | erase displayed memory (clear screen) | `cue.text = ''`
| 14 | carriage return (scroll lines up) | `cue.endTime = now(); 
cue.region = region; new VTTCue();`
| 15 | end of caption | `cue.endTime = now()`
| 16 | clear screen (erase display memory) | `cue.text = ''`
| 17 | tab offset 1/2/3 (add whitespace) | `cue.text += " " * 
num_space` (calculate numspace from tab offset as per 
https://dvcs.w3.org/hg/text-tracks/raw-file/default/608toVTT/608toVTT.html
 )

Analysing the table and browser implementations, it seems we already 
have most of the required functionality to do live captioning in the 
browser, bar the following:

* Chrome has a bug that when you change any object properties of an 
already rendered TextTrackCue, the rendering does not get updated.
* It's not possible to create cues with an open end time. The 
work-around for this right now is to use an end time that is really 
far into the future and then to update it when the end time is known.

Note that the `now()` function is a replacement for any start 
timestamp information that is available to the browser for the cue and
 may possibly be `video.currentTime`.

Please view or discuss this issue at 
https://github.com/w3c/webvtt/issues/319 using your GitHub account

Received on Friday, 14 October 2016 22:53:56 UTC