Re: [mediacapture-handle] Should the handle be an object? (#68)

To help drive collaboration between loosely-coupled apps, I've presented the following earlier today.

Assume:

```webidl
dictionary ContentHints {
  DOMString audio;
  DOMString video;
};

dictionary CropTargetData {
  DOMString name;
  CropTarget cropTarget;
  ContentHints suggestedContentHint;
};

dictionary CaptureHandleConfig {
  ...  // Pre-existing.

  // Suggested Content-Hint for uncropped video.
  ContentHints suggestedContentHints;

  // CropTargets of interest.
  sequence<CropTargetData> cropTargets;
};
```

Then a VC app could do something like this:
```js
const stream = navigator.mediaDevices.getDisplayMedia(constraints);
const [track] = stream.getVideoTracks();
const captureHandle = track.getCaptureHandle();

if (Object.keys(captureHandle?.cropTargets).length > 1) {
  // 1. Cycle through the crop-targets,
  // grabbing one frame with each.
  const thumbnails = GetThumbnails(track);

  // 2. Display an in-content picker.
  // The user chooses if they want to share
  // the entire tab, or only a specific part of it.
  const target = PromptUser(track, thumbnails);

  // 3. Apply user’s choice.
  track.cropTo(target);  // No-op if `target` is `undefined`.

  // 4. Transmit remotely.
  TransmitRemotely(track);
}
```

To put a cherry on top, in the `TransmitRemotely()` step, the VC app can - if it chooses - use the suggestedContentHint tied to the specific region.


This means that Teams, Zoom and Meet could all offer great **functionality** and **performonace** when capturing YouTube, Hulu, Slides, PowerPoint or anything else.


-- 
GitHub Notification of comment by eladalon1983
Please view or discuss this issue at https://github.com/w3c/mediacapture-handle/issues/68#issuecomment-1282898580 using your GitHub account


-- 
Sent via github-notify-ml as configured in https://github.com/w3c/github-notify-ml-config

Received on Tuesday, 18 October 2022 19:26:21 UTC