Re: [w3ctag/design-reviews] Captured Mouse Events (Issue #872)

> 1. What is the expected behavior for OSs that have no cursor - iOS, Android etc.?

Invisible "pointers" (touch, stylus) present a different security/privacy profile, because (a) they were not previously exposed to the capturing application, and (b) they involve exposing additional information that is similar to mouse-clicks. As such, they are outscoped, and we don't have concrete plans to extend the API to support them. (However, it seems like it will be possible if desired in the future, although likely subject to some additional permissions prompt.)

> 2. The intended effect assumes that the receiving UA will have the ability to change the lower layer OS cursors effects - is this possible? (For instance, we suspect this would be very diffcult on Windows.)

Changing the lower layer OS cursor is NOT required for any of the use-cases discussed. Rather, it is assumed that it is possible to *exclude* the OS cursor from capture. It is then straightforward to draw a custom cursor *into the video* that is either transmitted remotely or saved to disk. The local user would sitll see their own natural cursor locally; augementation for the local user is not part of the use-case discussed.

> 3. It is unclear how this feature will be available in different pixel densities, DPI, frame sizes etc.

Could you please help me understand the challenge here? Is it that the specification does not currently clarify whether the coordinates are used in terms of the captured frame's "pixel-space", running from (0, 0) to (max_x, max_y) as defined by `MediaStreamTrack.getSettings().width` and `MediaStreamTrack.getSettings().height`?

> 4. What other alternatives have you considered?

Exposure on the frames themselves rather than using dedicated events. This alternative is seen as complementary, and might be pursued in the future. The downside is that the frequency of the video might be quite low. For example, assume you want to capture at 5fps but render a smooth-looking video remotely by updaitng the cursor at 30fps.

> 5. How does this featuere interact with the a11y stack given they usually have thier own virutal cursor?

I think it should be up to the UA to decide whether, on the given platform with its particular implementation-limitations for the specific virtual cursor, the virtual cursor is treated as the real cursor, or is ignored.

> 6. Not a question but suggestion - we assumed that the spec is your explainer given the little information provided in the README that is linked as an explainer. Can you consolidate please?

If @fred-wang does not pick it up first, I can do so when I have more time to devote to this project.


-- 
Reply to this email directly or view it on GitHub:
https://github.com/w3ctag/design-reviews/issues/872#issuecomment-1680215689
You are receiving this because you are subscribed to this thread.

Message ID: <w3ctag/design-reviews/issues/872/1680215689@github.com>