Re: [mediacapture-region] What makes CropTarget special to require an asynchronous creation? (#17)

> Namely - what is this [incredible complexity](https://github.com/w3c/mediacapture-region/issues/17#issuecomment-1132095029) serving?

What I tried to say was that I do not see a big difference of UA implementation complexity/achievable performances between what Chrome implements and the proposed alternatives. The big difference I see is that Chrome current implementation puts some of the complexity on web pages in the name of this optimisation. In general, I think we should favour optimisations that are transparent to web pages.

> An application that decide to change the crop-target will often pause the track until it is informed that all subsequent frames are cropped to the new target. This can be noticeable to both the local user as well as remote users. This visible disruption should be minimzed.

I do not really understand in which conditions the application will pause the track, which I interpret as stop displaying/sharing it between the time cropTo is called and the time cropTo promise is resolved.

I can see a few related scenarios:

1. The web page wants to call getDisplayMedia to capture itself, apply cropping, then start displaying/sending the cropped track.
One natural approach here is to make just one getDisplayMedia call (something like `getDisplayMedia({ selfBrowserSurface: include, video : { cropTarget : myCropTarget } }`)).
This should simplify the web page workflow and allow some UA optimisations to get the smallest achievable latency.
Is it something that we should push to a spec as a way to reach consensus?

2. The web page is already capturing the tab and is displaying/sharing it. User decides to apply cropping through the web page UI.
In that case, calling cropTo after getDisplayMedia is the only way to implement this scenario.
This seems similar to when user requests to change camera (going from front to back camera say). I do not think web pages will pause the track in that scenario.
There is of course a need for cropTo to be fast enough so that the UI is responsive and that there is no excessive latency between user click and actual cropped frames to appear on screen.
In that case, it does not make sense to me for the web page to pause the track.

-- 
GitHub Notification of comment by youennf
Please view or discuss this issue at https://github.com/w3c/mediacapture-region/issues/17#issuecomment-1134262903 using your GitHub account


-- 
Sent via github-notify-ml as configured in https://github.com/w3c/github-notify-ml-config

Received on Monday, 23 May 2022 07:03:43 UTC