Re: [w3ctag/design-reviews] WebCodecs support for AV1 screen content coding tools (Issue #912)

While you are in the area could you please also review VideoEncoderConfig.contentHint by @Djuffin?
The explainer can be found [here](https://gist.github.com/Djuffin/c3742404b7c53ada227849c8b2b76b4c), [chromestatus entry here](https://chromestatus.com/feature/5103493249761280) and the tl;dr is
```
encoder.configure({
  codec: 'vp8',
  width: 1920, height: 1080, bitrate: 2_000_000, framerate: 5, 
  contentHint: 'detail'
}); 
```
Visually this should be similar to this older [webrtc sample](https://webrtc.github.io/samples/src/content/capture/video-contenthint/)

The relationship between the two is a bit complicated, bear with me.
* `contentHint` is codec-independent in the sense that it says "this is screensharing, treat it accordingly" and leaves the "how" to the codec. The "detail" and "motion" values already give a hint that the resolution and framerate are two parameters that influence this and how the encoder makes decisions on what to compromise on when hitting a target bitrate. See also [MediaStreamTrack Content Hints](https://w3c.github.io/mst-content-hint/#behavior-of-an-rtcpeerconnection) which calls this "degradation preference".
* `forceScreenContentTools` is AV1 specific and does not interact with resolution or framerate.

In WebRTC setting a MediaStreamTrack's contentHint treats it as "comes from screensharing" which implies screen content coding for AV1. Full details [here](https://bugs.chromium.org/p/webrtc/issues/detail?id=15484).


-- 
Reply to this email directly or view it on GitHub:
https://github.com/w3ctag/design-reviews/issues/912#issuecomment-1803544233
You are receiving this because you are subscribed to this thread.

Message ID: <w3ctag/design-reviews/issues/912/1803544233@github.com>