Re: [mediacapture-screen-share-extensions] Auto-pause capture when user switches captured content (#4)

Thinking a bit more, I am not sure the separation between `session tracks` and `surface tracks` is helping shape this API.

Let's look at the following two scenarios:
1. User agent U1 is exposing a switch surface UX and user clicks on it. User is expecting that the new surface content will be rendered where the past track content was rendered.  It seems reasonable that the same track exposes the media content and that no new track is exposed: session track model seems well suited.

2. User agent U2 is exposing an add/remove surface UX. User first adds a surface B and then removes the original surface A. I would think user is expecting the app to react to the new track somehow by using a new video element for rendering. This seems inline with the user agent exposing a new track for B and ending the original track A: surface track model seems well suited.

Given this, and given UX in that area is relatively new, I am not sure we can design an API that specifies a particular flow. Having an API that exposes new tracks and having a requirement that video frames of the switching surface do not get provided to sinks until some event/callback actually runs might be good enough for now. Plus some guidelines...

That said, AFAIK, the only thing UAs are doing right now is scenario 1 above. And this is what the initial message of this issue is describing.
Based on this, I would suggest trying to fix this and only this for now by specifying the requirement that video frames of the switching surface do not get provided to sinks until some event/callback actually runs. I would tend to use the `configuration change`/`deviceId` combo for that as it does not require new API surface.

-- 
GitHub Notification of comment by youennf
Please view or discuss this issue at https://github.com/w3c/mediacapture-screen-share-extensions/issues/4#issuecomment-2401497963 using your GitHub account


-- 
Sent via github-notify-ml as configured in https://github.com/w3c/github-notify-ml-config

Received on Wednesday, 9 October 2024 07:02:49 UTC