[media-and-entertainment] Compositable Video Content (#103)

AdamSobieski has just created a new issue for https://github.com/w3c/media-and-entertainment:

== Compositable Video Content ==
In addition to sets of video tracks in containers and streaming solutions representing alternative renditions of content, let us consider describing sets of video tracks where those tracks are intended to be [blended](https://en.wikipedia.org/wiki/Blend_modes) together by clients per [alpha compositing](https://en.wikipedia.org/wiki/Alpha_compositing).

Alpha composting is, today, most often performed server-side before content is streamed or broadcast by video production software.

Uses of client-side video layer compositing would include:

1. Some video layers could be delivered with a variable bitrate, e.g., news anchors speaking, while other video layers could be delivered with an assured fidelity, e.g., text overlays, lower third content, and news tickers.

2. Video players could automatically select which video layers to composite together on behalf of end-users. A client-side system could, for instance, automatically select a foreground layer from a set of alternatives based on a user's preferred language.

3. Web pages could allow end-users to make use of menus to dynamically select which video layers to composite together. End-users could, for instance, select which variety of news that they would prefer for a news ticker composited atop other video content.

4. Client-side compositable video layers could be a part of solutions for enabling interactive time-based hypermedia, e.g., [hypervideo](https://en.wikipedia.org/wiki/Hypervideo). Foreground layers could be capable of having hypertext selected from them or of otherwise being interacted with, e.g., clicked upon. As considered, this type of video layer might provide bounding boxes or silhouettes, perhaps with identifiers unique to the video container or stream.

A useful implementation concept could be that of "trees of video tracks". With such trees, we could express sets of video tracks for alpha compositing where some of these layers could have alternative renditions or reflowed renditions. Recursively, some of these layers could be intended to be rendered using client-side video layer compositing.

Perhaps, one day, video containers and streaming technologies will be able to combine client-side compositability with [reflowability](https://github.com/w3c/media-and-entertainment/issues/102) to provide video producers and consumers with new features and functionalities.

Thank you. I look forward to discussing these ideas with you.

Please view or discuss this issue at https://github.com/w3c/media-and-entertainment/issues/103 using your GitHub account


-- 
Sent via github-notify-ml as configured in https://github.com/w3c/github-notify-ml-config

Received on Monday, 21 August 2023 19:45:34 UTC