Re: [media-and-entertainment] Reflowable Video Content (#102)

Quoting from [Section 4.3.4.2.1](https://datatracker.ietf.org/doc/html/draft-pantos-http-live-streaming-23#section-4.3.4.2.1):

> If an EXT-X-STREAM-INF tag contains a RESOLUTION attribute and a VIDEO attribute, then every alternative video Rendition MUST have an optimal display resolution matching the value of the RESOLUTION attribute.

It appears that the EXT-X-STREAM-INF tag should contain a VIDEO attribute but not a RESOLUTION attribute. In this way, every alternative video Rendition could specify a distinct value for its RESOLUTION attribute.

Also from [Section 4.3.4.2.1](https://datatracker.ietf.org/doc/html/draft-pantos-http-live-streaming-23#section-4.3.4.2.1):

> Every alternative Rendition associated with an EXT-X-STREAM-INF tag MUST meet the constraints for a Variant Stream described in [Section 6.2.4](https://datatracker.ietf.org/doc/html/draft-pantos-http-live-streaming-23#section-6.2.4).

Quoting from [Section 6.2.4](https://datatracker.ietf.org/doc/html/draft-pantos-http-live-streaming-23#section-6.2.4):

> Each Variant Stream MUST present the same content.

Under discussion are variations with potentially different layouts. That is, the streams, while _roughly_ the same content, might differ in terms of the layout of widgets (or of other content) or in terms of the layout of text content.

To deliver these features for Web scenarios, there would be some work with respect to these topics on the HTML5 side.

The [Mozilla Developer Network (MDN)](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/source) describes uses of the `media` attribute on the `<source>` element as follows:

> Allowed if the source element's parent is a [`<picture>`](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/picture) element, but not allowed if the source element's parent is an [`<audio>`](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/audio) or [`<video>`](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/video) element.
>
> [Media query](https://developer.mozilla.org/en-US/docs/Web/CSS/CSS_media_queries) of the resource's intended media.

However, the [HTML5 living standard document](https://html.spec.whatwg.org/multipage/embedded-content.html#attr-source-media), describes uses of the `media` attribute on `<source>` elements as follows:

> The media attribute may also be present. If present, the value must contain a [valid media query list](https://html.spec.whatwg.org/multipage/common-microsyntaxes.html#valid-media-query-list). The user agent will skip to the next [source](https://html.spec.whatwg.org/multipage/embedded-content.html#the-source-element) element if the value does not [match the environment](https://html.spec.whatwg.org/multipage/common-microsyntaxes.html#matches-the-environment).

> The [media](https://html.spec.whatwg.org/multipage/embedded-content.html#attr-source-media) attribute is only evaluated once during the [resource selection algorithm](https://html.spec.whatwg.org/multipage/media.html#concept-media-load-algorithm) for [media elements](https://html.spec.whatwg.org/multipage/media.html#media-element). In contrast, when using the [picture](https://html.spec.whatwg.org/multipage/embedded-content.html#the-picture-element) element, the user agent will [react to changes in the environment](https://html.spec.whatwg.org/multipage/images.html#img-environment-changes).

It appears that the HTML5 living standard discusses source selection for `<video>` elements containing `<source>` elements with `media` attributes as occurring during initialization, using the resource selection algorithm.

While these topics of resizing and dynamic environments appear to have been considered with respect to `<picture>` elements, the capabilities for video elements to switch streams during or as a result of resizing events or other environment changes would (seemingly) be new ground with respect to HTML5.

To deliver these capabilities, in my opinion, we would want both the capability of describing these features for video envelopes (which could be, for HLS, `alternative renditions`) and the capability to use HTML5 to express different stream sources with CSS Media Queries in the `media` attributes on `<source>` elements contained in `<video>` elements. Then, as a result of resizing video elements or other environment changes, video elements could switch streams and this would enable reflowable video content.

-- 
GitHub Notification of comment by AdamSobieski
Please view or discuss this issue at https://github.com/w3c/media-and-entertainment/issues/102#issuecomment-1685109669 using your GitHub account


-- 
Sent via github-notify-ml as configured in https://github.com/w3c/github-notify-ml-config

Received on Saturday, 19 August 2023 20:46:36 UTC