Re: [w3ctag/design-reviews] Captured Surface Control (Issue #962)

> We think this seems like a generally useful feature

That's great to hear!

> The explainer should discuss the alternative design of having the page cooperate, and accept dedicated events from the capturing process. We think there are both upsides and downsides to that option that deserve exploration.

I have now added a discussion select alternatives to the explainer.

> The two interactions that are considered are scrolling and zooming. Is that list exhaustive?

For the time being - yes.

Apple's represenative, Youenn, suggested adding pinch. No Web developers have requested this feature, so we are leaving this as a potential extension. But note that the current API shape does not prevent such future extensions.
* We could, in the future, define `capturePinch(element)`.
* We could, in the future, transition to `forwardGesutres(element, gestures)`, where the second argument is a dictionary of relevant gestures.
* Other API shapes would be possible.

Note: We intentionally **exclude** any interaction like clicking, delivering keystrokes, etc. We have no plans of ever extending the API to cover such gestures.

> Are these uniformly safe to do? Are there not occasions where scrolling results in changes to things like form elements?

Web applications can attach any meaning to any user action, and that property is desirable and necessary to retain - the user expects scrolling to work identically when delivered from the capturing application; always, not just when it's a simple scroll. A concrete example is Google Maps, where scrolling results in change the region of the map being displayed, triggering the fetching of new assets, etc. Or think how Apple's main page often uses fancy animations of laptops opening and closing when scrolling.

We believe that this risk is sufficiently mitigated by the (1) pre-existing safeguards associated with screen-sharing to begin with, by (2) the additional permission prompt involved, and (3) by the steps taken to ensure that only the user's immediate interaction with the capturing application can trigger scroll-forwarding to the captured application.

> That could require a change of focus before sending the events in

Mandating change of focus could break the experience for the user and subvert their expectation, that the scroll delivered on the capturing application's preview tile, would end up eliciting the **exact** same behavior on the captured surface, as though it were delivered directly there.

> There seems to be some heightened permissions UX being contemplated here. It's not clear to us what would be different from a regular screen capture. It would be helpful if the explainer could show a proof of concept that highlights those differences.

When users are currently asked to grant permission to capture a tab/window/screen, they are used to a specific interpretation. Before elevating this permission to something new - capture plus scroll plus zoom - an additional prompt is required. User agents are free to infer this heightened permission using any heuristic, and may change that based on how user expectations evolve over time. For the time being, Chrome intends to use a run of the mill permission prompt, and to use some extra UX to clarify to the user that this permission is active. This is neither mandated by the spec, nor do we guarantee that Chrome will retain this particular UX.

-- 
Reply to this email directly or view it on GitHub:
https://github.com/w3ctag/design-reviews/issues/962#issuecomment-2429483359
You are receiving this because you are subscribed to this thread.

Message ID: <w3ctag/design-reviews/issues/962/2429483359@github.com>