Re: [mediacapture-screen-share] Auto-pause capture when user switches captured content (#255)

As [mentioned](https://docs.google.com/presentation/d/1OLivIHYOLKjsFnINq2nibBk9RtwCQjkEyIazyiFzhh0/edit#slide=id.g22ae25380eb_1_0) during the June meeting of the Screen Capture CG, the issue presented in this thread might be tackled along with a different one, using a mechanism other than auto-pause.

Other issue:
1. Application calls getDisplayMedia() and obtains a MediaStream containing a video MediaStreamTrack `vid`.
2. Application checks `vid.getSettings().displaySurface` and detects that the user chose to share a specific surface type (say `'monitor'`).
3. User changes the capture to some other surface type (say `'window'`).
4. Application attempts to call some method on `vid`, that behaves differently on the first surface type compared to the second. (Say throws an error.)

Reasonable applications deployed today are likely to only perform the check of surface type immediately after gDM resolves, and set cascading state based on it (modify user-facing controls that can later trigger step 4). Such applications would not have been written to check again before step 4, because dynamic switching across surface types did not yet exist.

The backwards-compatible solution I [presented](https://drive.google.com/file/d/1OW6kcQbX5XQLqJSFaDabtUJTJcRiWZU6/view) during the Screen Capture CG meeting is as follows:
```js
const controller = new CaptureController();
controller.enableDynamicSwitching('switch', (event) => {
  const videoElement = document.getElementById('myVideoElement');
  videoElement.srcObject = event.mediaStream;
  // And/or outgoing peer connection; imagine whatever you find relevant.
});
const stream = await navigator.mediaDevices.getDisplayMedia({ controller });
...
```

The spec would then say that the user agent MAY constrain dynamic-switching options based on whether `enableDynamicSwitching()` was called (before gDM was called), thereby ensuring that cross-type surface-switching only happens to apps that can handle it gracefully.

Note how the solution provides auto-pause "for free" by virtue of creating a new stream (and new tracks).

-- 
GitHub Notification of comment by eladalon1983
Please view or discuss this issue at https://github.com/w3c/mediacapture-screen-share/issues/255#issuecomment-1610999319 using your GitHub account


-- 
Sent via github-notify-ml as configured in https://github.com/w3c/github-notify-ml-config

Received on Wednesday, 28 June 2023 08:35:36 UTC