Re: [mediacapture-screen-share] A CaptureController object for getDisplayMedia() (#230)

> ```js
> const stream = await navigator.mediaDevices.getDisplayMedia({controller});
> ```

Would this be `getDisplayMedia({video: ..., controller: controller})`?

> ```js
> await controller.focus();
> ```

Do we really want to inform the capturer that the capturee has gained focus?

The capturing application can observe that it has, itself, lost focus. But it can't tell if the user has `alt-tab`-ed to another native app, switched to another tab, etc. And I think that's enough.

Maybe you intention here is to convey an error message back to the application when focus() fails? Would this be actionable and useful enough to merit the complexity and privacy-concerns entailed?

> Without a controller, getDisplayMedia would bring-to-front with focus like today for backwards compatibility.

On the one hand, I really like how this requires an explicit decision from new code while still retaining the established UA behavior old code. Well done there!

On the other hand, I am concerned about specifying defaults.

We have two cases to discuss defaults for:
1. getDisplayMedia() is called **without** a controller.
2. getDisplayMedia() is called **with** a controller, but the application either fails to call focus(), or does so unreasonably late.

I think we should start out leaving the defaults here up to the UA, and circle back to this if we manage to get consensus. This is reasonable because:
1. **Without** a controller - the status quo is maintained. No new commitments, but in practice we all do the same (Chromium and Firefox both default to focusing; Safari has not yet implemented).
2. **With** a controller, but failing a **timely** call to focus() - I think an MVP can reasonably neglect to specify it.
   a. Applications that forget to call `focus()` are buggy and should be fixed.
   b. Applications that call `focus()`, but CPU hiccups render the call too late - this will be very uncommon, and inconsistent cross-browser behavior here is an insignificant concern to developers and users.

> This is UA territory, I am not sure the current spec says anything about this.
> A note in the spec would probably be good here.

If I am reading @youennf correctly here, then we're in agreement; see above.

> I am not a big fan of a timer here.

The timer is a necessary evil, to remove the sting from this hypothetical malicious code:
```js
  await ...getDisplayMedia({video: ..., controller: controller});
  busyWait(/*seconds=*/5);
  controler.setFocusBehavior("focus-captured-surface");
```

Note that it doesn't have to be malicious. If a CPU hiccup or buggy code happens to cause the shared surface to be focused at an arbitrary, unexpected point in the future, it will surprise and annoy the user. It might also produce an unintended interaction with the focused surface, that might have been intended for another page.

Speaking of which - we should also require that focus() be rendered no-op if the capturing page is not the focused one by the time focus() is evaluated. I believe my current draft of [Conditional Focus](https://wicg.github.io/conditional-focus/) does that, but otherwise, we could change it.

> controller.focus() is a bit unclear what is being focused, capturee or capturer. Any idea how to make the name clearer?

I agree that there is unclarity here. Chrome has gone with the following, and I think it's a good place to start:
```webidl
// Previously on the track itself, now on the controller:
undefined focus(CaptureStartFocusBehavior focus_behavior);

enum CaptureStartFocusBehavior {
  "focus-captured-surface",
  "no-focus-change"
};
```

I'd just change `focus()` to `setFocusBehavior()` and I think we're set. We'll also have a credible path, post-MVP, to allowing focus-behavior to be set before getDisplayMedia(), which Jan-Ivar was previously interested in, so that should be good too.

> It might make sense to allow UA to override the page asking not to focus. This is probably important to state something like this given this is observable to JS. focus could reject with an error of type NotAllowedError for instance.

I'm OK with that.

> It is also worth pointing out that not focusing increases the risk of mis-selection by the user, so particular care should be brought there by UA to limit this risk.

There are two risks here:
1. The risk that mis-selection *will* occur.
2. The risk *once* mis-selection has occurred.

Chrome's approach to this problem was to reduce the first risk. We did that by introducing a preview for tabs (we'd already had them for windows and screens). With that in place, the second risk has become far less relevant.


-- 
GitHub Notification of comment by eladalon1983
Please view or discuss this issue at https://github.com/w3c/mediacapture-screen-share/issues/230#issuecomment-1234049897 using your GitHub account


-- 
Sent via github-notify-ml as configured in https://github.com/w3c/github-notify-ml-config

Received on Thursday, 1 September 2022 10:02:01 UTC