Re: [mediacapture-screen-share] Identification of Captured Application By Capturer (#166)

I've been discussing this with @jan-ivar, and my understanding of his current position is that he would support [this proposal](https://wicg.github.io/capture-handle/)[*] if we **add** the ability for some **basic messages** to be sent capturer->capaturee. This sounds like a good idea to me, and I am happy to resume the discussion accordingly.

The proposal would then comprise two parts:
1. We keep [setCaptureHandleConfig](https://wicg.github.io/capture-handle/#set-capture-handle-config), [getCaptureHandle](https://wicg.github.io/capture-handle/#capture-handle-via-getcapturehandle), etc. This allows sophisticated cooperation between closely collaborating applications.
2. We add a mechanism for applications to declare which basic messages (prev/next/first/last) they support, a mechanism for applications to register a handler for such messages, and a mechanism for a capturing app to send these messages to the captured application.

### Overview of the added basic-messaging capabilities:

#### Shared capturer/capturee:
* Define a small set of strings, representing actionable messages, that can be sent capturer->capturee.
  * Possibly `{"first", "prev", "next", "last"}`.
  * Pre-bikeshedding name: `CaptureActions`.

#### Capturee-side:
* Add a control allowing declaring supported actions.
  * Possibly `MediaDevices.setSupportedCaptureActions()`.
* Add a control for setting a handler for incoming messages.
  * Incoming messages are received as an event that exposes the single action sent.
  * For multiple concurrent capturers, we could later add a `source` field in the event. At the moment, I don't think this is an important edge case, so I'd rather leave this initially unaddressed.

#### Capturer-side:
* Add a control, exposed on the track, for reading the actions supported by the application running in the captured surface.
* Add a control, exposed on the track, for sending a message to the captured application's handler.
  * Rate-limit messages.
  * Only allow valid `CaptureActions` messages, but do not verify that the message was declared as support, since there's no harm in letting them through and no reasonable attack enabled through them if they are just ignored. (Devil's advocate - what if an application only declares one action as supported and just assumes that's the only message it's ever getting? Counter - then that application is defective.)

#### Noteworthy
* The capturee is still unaware that it's captured, unless the capturer intentionally alerts it to the fact.
* Declaring supported actions allows the capturer to expose only relevant user-facing controls, and do so without really knowing what it's capturing. For example, a video-conferencing application authored tomorrow and never updated later, could still expose only prev/next buttons when capturing Hypothetical-Slides-Deck, but expose first/prev/next/last when capturing Hypothetical-Video-App, even if both of these captured apps were written long after the VC app, and have never set up collaboration specifically with it.

@jan-ivar: Have I accurately represented your position? Do you agree with the general approach I suggest here for basic messaging?
@youennf, @aboba and everybody else: What do you think? Would you support the adoption of [Capture Handle](https://wicg.github.io/capture-handle/) if we add these mechanisms?

-- 
GitHub Notification of comment by eladalon1983
Please view or discuss this issue at https://github.com/w3c/mediacapture-screen-share/issues/166#issuecomment-1012503381 using your GitHub account


-- 
Sent via github-notify-ml as configured in https://github.com/w3c/github-notify-ml-config

Received on Thursday, 13 January 2022 20:48:02 UTC