Re: [mediacapture-handle] Should the handle be an object? (#68)

To help drive collaboration between loosely-coupled apps, I've presented the following earlier today.


dictionary ContentHints {
  DOMString audio;
  DOMString video;

dictionary CropTargetData {
  DOMString name;
  CropTarget cropTarget;
  ContentHints suggestedContentHint;

dictionary CaptureHandleConfig {
  ...  // Pre-existing.

  // Suggested Content-Hint for uncropped video.
  ContentHints suggestedContentHints;

  // CropTargets of interest.
  sequence<CropTargetData> cropTargets;

Then a VC app could do something like this:
const stream = navigator.mediaDevices.getDisplayMedia(constraints);
const [track] = stream.getVideoTracks();
const captureHandle = track.getCaptureHandle();

if (Object.keys(captureHandle?.cropTargets).length > 1) {
  // 1. Cycle through the crop-targets,
  // grabbing one frame with each.
  const thumbnails = GetThumbnails(track);

  // 2. Display an in-content picker.
  // The user chooses if they want to share
  // the entire tab, or only a specific part of it.
  const target = PromptUser(track, thumbnails);

  // 3. Apply user’s choice.
  track.cropTo(target);  // No-op if `target` is `undefined`.

  // 4. Transmit remotely.

To put a cherry on top, in the `TransmitRemotely()` step, the VC app can - if it chooses - use the suggestedContentHint tied to the specific region.

This means that Teams, Zoom and Meet could all offer great **functionality** and **performonace** when capturing YouTube, Hulu, Slides, PowerPoint or anything else.

GitHub Notification of comment by eladalon1983
Please view or discuss this issue at using your GitHub account

Sent via github-notify-ml as configured in

Received on Tuesday, 18 October 2022 19:26:21 UTC