Re: [mediacapture-surface-control] Is gesture forwarding tied to capture controller or to MediaStreamTrack or to DOM objects? (#45)

Risks to end-users of wheel-jacking comes ahead of inconveniencing web developers in the [§ 1.1. Put user needs first (Priority of Constituencies)](https://w3ctag.github.io/design-principles/#priority-of-constituencies) (consent doesn't really change this).

> Without libraries, “backwarding” consists merely of invoking myDrawOnClick() from video.onclick instead of from canvas.onclick – a possible change.

No, what I meant with forwarding/backwarding was dealing with an underlying interactive element. But it looks like I wrongly assumed this library was written to overlap another interactive element, but it sounds like it's not designed for that even and is stealing all input for itself, is that right? If so, that's not a very compelling limitation, for the following reason:

From a logical and conceptual perspective, wheel forwarding to a video preview of a live tab-capture is clearly interactive, and clearly a feature of the video element presenting it. This seems obvious down to the details of:
1. coordinates, which need to align with the offset and scaling of the video presentation. 
2. Resizing or re-layout of the video element will affect the necessary transformation of coordinates to land in the right place in the captured tab.
3. proposed restrictions, like `video.srcObject` matching a the right capture, and forwarding only working while it is playing (meaning end-of-forwarding should probably be tied to end of playback, or end-of-track at least, _**not**_ end of capture which is different due toe track.cloning).

If the only reason to put it elsewhere is a library not designed to overlap an interactive element, then that is not a good reason.

I'm not opposed to solving the annotation use case, but it's secondary to the primary design of this API in my view.

To illustrate: worst case, the overlay problem seems solvable by something like this:
```js
await videoElement1.forwardGestures(true, {overlay: element}});
```
If we can move the window of discussion there, then we can probably discuss whether `{overlay: div}` argument is truly needed or if the user agent can figure out the overlapping element(s) itself.

-- 
GitHub Notification of comment by jan-ivar
Please view or discuss this issue at https://github.com/w3c/mediacapture-surface-control/issues/45#issuecomment-2613084231 using your GitHub account


-- 
Sent via github-notify-ml as configured in https://github.com/w3c/github-notify-ml-config

Received on Friday, 24 January 2025 17:51:27 UTC