Re: [mediacapture-surface-control] Is gesture forwarding tied to capture controller or to MediaStreamTrack or to DOM objects? (#45)

After the discussion in the WG meeting, I see that this is Google's position:

1. The use case concerns 3 entities:
    * An element that events are forwarded from (usually called overlay element in the discussions)
    * A captured tab where events are forwarded to
    * Some element used for rendering the capture (typically a video element or canvas). In the very simplest cases this element can be the same as the overlay, but in general it is not (core use cases like annotations generally require them to be separate).
2. Since the only constant in the 3 entities mentioned above is the captured tab, the method should be a member of the object that best represents the captured tab. That is, the CaptureController. We are not open to changing this, because while overlays and rendering methods can change, the fact that the events are forwarded to a captured tab cannot change. Putting the method elsewhere would risk an combinatorial explosion depending on how we add support for other features in the future. For example, suppose we add `forwardXXX` and `forwardYYY`, attributes to get the current status, and support for rendering via the `WWW` and `ZZZ` mechanisms. If we tie the method to a rendering element then we need to define _N_ methods in _M_ rendering mechanisms. Keeping it in CaptureController ensures that there is a single place for any extension that may come up in the future. We are not open to changing our position on this.

3. Ensuring that there is coincidence (hit zone) between whatever is used for rendering and the overlay element has been presented as in important issue. We are willing to compromise here, as long as we express the requirement as a MAY or SHOULD in the spec, so as to allow different UAs to experiment in this area. We are also open to letting the application express intent by optionally passing a rendering element (canvas or video) as input, so that the UA can use that as input to determine the intended hit zone. The UA may also try to determine the presence of a rendered frame in the location of the overlay element without this extra optional input.

4. Support for touch events has been presented as a potential requirement. We see this as an interesting extension and are open to adjust the API surface to make it extensible to support this, as long as we make it possible to separate wheel and touch events. Our initial preference is the `forwardGestures` proposal, which allows turning forwarding on or off by passing a boolean option for each potential event type (e.g., `controller.forwardGestures({overlay: myoverlay, touch: true})` to enable touch forwarding from `myoverlay` to `controller` (and false to disable it). We are also open to alternatives, such as a `forwardTouch` method, a separate synchronous `updateGestureForwarding()` method, or any other alternatives, provided that the core use case of enabling and disabling forwarding from one overlay to one captured tab is well supported.

5. Support for multiple previews/overlays has been presented as a potential requirement. While this is not a core use case, we are open to having an API surface that can be extended to support this. Again, our preference is the `forwardGestures` proposal, which allows passing an overlay and a boolean to turn forwarding on or off for that overlay. The CaptureController can keep track of all the overlays that are potentially forwarding and of the status of the event types to be forwarded for each overlay. We are also open to alternatives, provided that the core use case of enabling and disabling the forwarding from one overlay to one captured tab is well supported.


-- 
GitHub Notification of comment by guidou
Please view or discuss this issue at https://github.com/w3c/mediacapture-surface-control/issues/45#issuecomment-2698435190 using your GitHub account


-- 
Sent via github-notify-ml as configured in https://github.com/w3c/github-notify-ml-config

Received on Tuesday, 4 March 2025 17:38:58 UTC