- From: Elad Alon via GitHub <sysbot+gh@w3.org>
- Date: Tue, 30 Aug 2022 17:09:45 +0000
- To: public-webrtc-logs@w3.org
> 1. If there is tight coupling between capturer and capturee, this API is not needed, or more precisely this is just a small optimization, so low in priority. I'd phrase it differently. * If there is NO tight coupling, then this API is the only way to accomplish hinting. * If there IS tight coupling, then this API is still VERY helpful, because: * Instantenous hinting with no bandwidth overhead. * No need for each capturer/capturee pair to reinvent hinting. > 2. In the short term, content hint can already be provided as part of the handle value. Only between tightly coupled apps, since the `handle` is not a structured field, so it won't be clear where the hint lies and where other information is stored. For example, one capturee could set it as "session: 142, hint: HINT" while another capturee sets it as "<hint>HINT</hint>", and the capturer might know neither of them. My proposed API solves this. > Let's take the time to do the best design we can. Let's always do our best. Time-wise, how long should this take? > 3. In another short term, the UA could use that content hint automatically (at least in RTCPeerConnection). Whether UAs use such an optimization seems out of scope for our discussion, as Web-devs with a stake have already agreed that they need more than what automatic optimization can offer. > 4. If there is no tight coupling between capturer and capturee, how is capturer supposed to interpret capturee content hint? No coupling necessary because it's a structured field that can be passed directly into the track's `contentHint` field. (Debtable what to do if a capturee tries to set a value that's not a legal contentHint - throw, ignore-and-allow or ignore-and-no-op.) See [slide 50](https://www.w3.org/2022/04/26-webrtc-minutes.html#t06). > Should it trust it or not? Up to the capturer to decide if it should apply hints from trusted sources only. My proposal is to trust the capturee, because there is no incentive for the capturee to lie - they'd only be annoying their own users, which is not a good business model or attack vector. (I can foresee discussions of "Docs could use misleading hints that only Meet knows to ignore" and I just don't find them convincing. But if someone has such a concern, then let them ignore untrusted hints, and that's that.) > Maybe capturee input is only valid in a given context (say encoder is VP8) but is not good for other contexts (say encoder is H264). 1. I don't think such hints currently exist. Or do you want to file a bug against the [MST Content Hint](https://www.w3.org/TR/mst-content-hint/) working draft? I see @alvestrand is an editor. 2. Suppose these exist, or will exist in the future - then we'll find a structured way to provide hint-per-codec. > 5. In a world of VideoFrames, it seems this hint could be exposed as a VideoFrame metadata. Capture Handle specifies events already. These are absolutely necessary, because the captured tab can be navigated. Let's take advantage of that mechanism rather than reinvent it in a new context. > 6. This API is not scalable as it is. Do you have a better suggestion? > For instance, maybe capturee will only provide a content hint that is meaningful after cropping is done but some capturers may not do cropping. APIs can be misused. If you suggest a fool-proof API, I'll be happy to adopt it. Otherwise, I don't think "this is not 120% perfect" is a reason to avoid progress. > 7. I wonder whether handle should be an object (structure clonable or something like that) instead of a string. This way, the handle could contain some structured information (including CropTarget, content hints and so on). Adding structure is precisely what this proposal is all about. -- GitHub Notification of comment by eladalon1983 Please view or discuss this issue at https://github.com/w3c/mediacapture-handle/issues/35#issuecomment-1231939151 using your GitHub account -- Sent via github-notify-ml as configured in https://github.com/w3c/github-notify-ml-config
Received on Tuesday, 30 August 2022 17:09:47 UTC