Using WebVTT For Bullet Chat

I’m interested in whether bullet chat comments can be represented using WebVTT as I’m leading the development of Web Video Map Tracks (WebVMT) [1], which started as an extension to WebVTT but eventually branched into a separate format. I’m keen to understand the bullet chat similarities and differences as I think there's an overlap with WebVMT, particularly with CSS style handling.

WebVTT supports live streaming and the WHATWG spec gives an example of this [2] with WebVTT syntax. WebVMT has also been designed to handle live use cases from the outset and this is an active topic for development in the WICG DataCue activity [3].

* Similarities

If I've understood correctly, bullet chat comments are synchronised with media - like subtitles. Hence, they share a lot of key characteristics with HTML’s TextTrackCue [4] - a start time, end time and content - which can be represented by WebVTT. Each comment can be individually styled using CSS to control font size, colour, etc. and the panel in which it is displayed determines display area, scrolling direction, maximum texts displayed concurrently, etc.

TextTrackCue is designed to synchronise text content with a timeline, which is generally (though not necessarily) associated with a media stream, so I see nothing to prevent ‘live' subtitles being added in real time to a web page. When the user submits a comment, the start time can be set to the current stream time, the content is set by the user’s text and the end time can be set to the start time plus the comment’s duration, e.g. scroll time. My understanding is that this is functionally the same as bullet chat.

* Differences

I note that there can be interaction with a bullet chat comment, e.g. a viewing user can hover or click to slow or stop a comment, but this only affects the viewer’s own display and not the broadcast content seen by other viewers. This could be handled by updating the end time of an existing cue in the viewer’s user agent in the way proposed in WICG DataCue issue #9 [5].

One way of implementing bullet chat is for the user to post their comments with a ‘live’ timestamp to a web server which aggregates them into a database and then streams all users’ content back in VTT format [6] alongside the media stream. Comments would be synchronised to the correct media time, merged with other users’ content, and interactions could be handled in the user agent without affecting the broadcast VTT stream as they only affect the local rendering of the (common) bullet chat content.

Rob Smith

Away Team
www.awayteam.co.uk <http://www.awayteam.co.uk/>

[1] https://w3c.github.io/sdw/proposals/geotagging/webvmt/ <https://w3c.github.io/sdw/proposals/geotagging/webvmt/>
[2] https://html.spec.whatwg.org/multipage/media.html#best-practices-for-metadata-text-tracks <https://html.spec.whatwg.org/multipage/media.html#best-practices-for-metadata-text-tracks>
[3] https://github.com/WICG/datacue <https://github.com/WICG/datacue>
[4] https://html.spec.whatwg.org/multipage/media.html#texttrackcue <https://html.spec.whatwg.org/multipage/media.html#texttrackcue>
[5] https://github.com/WICG/datacue/issues/9 <https://github.com/WICG/datacue/issues/9>
[6] https://w3c.github.io/webvtt/ <https://w3c.github.io/webvtt/>