Re: Using WebVTT For Bullet Chat

I think you're right and webvtt should be able to solve that problem.

On a side note, why did you have to define a completely new format for the
mapping use cases? Want it possible to define a new metadata type for track
and reuse webvtt? I'm asking because webvtt was built with the idea of
being extensible for all use cases for time aligned data. If there's a
limitation to the extensibility, it should get addressed.

Cheers,
Silvia.

On Wed, Jan 22, 2020, 3:45 AM Rob Smith <rob.smith@awayteam.co.uk> wrote:

> I’m interested in whether bullet chat comments can be represented using
> WebVTT as I’m leading the development of Web Video Map Tracks (WebVMT) [1],
> which started as an extension to WebVTT but eventually branched into a
> separate format. I’m keen to understand the bullet chat similarities and
> differences as I think there's an overlap with WebVMT, particularly with
> CSS style handling.
>
> WebVTT supports live streaming and the WHATWG spec gives an example of
> this [2] with WebVTT syntax. WebVMT has also been designed to handle live
> use cases from the outset and this is an active topic for development in
> the WICG DataCue activity [3].
>
> * Similarities
>
> If I've understood correctly, bullet chat comments are synchronised with
> media - like subtitles. Hence, they share a lot of key characteristics with
> HTML’s TextTrackCue [4] - a start time, end time and content - which can be
> represented by WebVTT. Each comment can be individually styled using CSS to
> control font size, colour, etc. and the panel in which it is displayed
> determines display area, scrolling direction, maximum texts displayed
> concurrently, etc.
>
> TextTrackCue is designed to synchronise text content with a timeline,
> which is generally (though not necessarily) associated with a media stream,
> so I see nothing to prevent ‘live' subtitles being added in real time to a
> web page. When the user submits a comment, the start time can be set to the
> current stream time, the content is set by the user’s text and the end time
> can be set to the start time plus the comment’s duration, e.g. scroll time.
> My understanding is that this is functionally the same as bullet chat.
>
> * Differences
>
> I note that there can be interaction with a bullet chat comment, e.g. a
> viewing user can hover or click to slow or stop a comment, but this only
> affects the viewer’s own display and not the broadcast content seen by
> other viewers. This could be handled by updating the end time of an
> existing cue in the viewer’s user agent in the way proposed in WICG DataCue
> issue #9 [5].
>
> One way of implementing bullet chat is for the user to post their comments
> with a ‘live’ timestamp to a web server which aggregates them into a
> database and then streams all users’ content back in VTT format [6]
> alongside the media stream. Comments would be synchronised to the correct
> media time, merged with other users’ content, and interactions could be
> handled in the user agent without affecting the broadcast VTT stream as
> they only affect the local rendering of the (common) bullet chat content.
>
> Rob Smith
>
> Away Team
> www.awayteam.co.uk
>
> [1] https://w3c.github.io/sdw/proposals/geotagging/webvmt/
> [2]
> https://html.spec.whatwg.org/multipage/media.html#best-practices-for-metadata-text-tracks
> [3] https://github.com/WICG/datacue
> [4] https://html.spec.whatwg.org/multipage/media.html#texttrackcue
> [5] https://github.com/WICG/datacue/issues/9
> [6] https://w3c.github.io/webvtt/
>
>