Re: [mediacapture-region] What makes CropTarget special to require an asynchronous creation? (#17)

> And all of this wonderful complexity is in the service of...?

@eladalon1983, when looking at this API without any particular implementation in mind, the simplest approach I can think of is this one (with A the getDisplayMedia caller, B the CropTarget creator, C the cropper):
B serializes CropTarget information as bytes to A by calling postMessage, which sends them to C by calling cropTo. C uses the provided bytes to do cropping.

You are bringing to the discussion security concerns.
It would be good to understand more precisely these concerns, after all A is only removing info by calling cropTo.
One approach is for C to validate/enrich the bytes with the help of B through an IPC exchange (some APIs do that A -> C -> B dance).
An alternative is to hash/encrypt the bytes so that C can validate them without B's help.

You are then bringing to the discussion performance concerns.
The first question is of course understanding what are these concerns. Is it loosing 1 or 2 frames or one second of frames?
The typical use case I see is a user clicking on UI to trigger cropping. A latency of a few frames seems fine, 1 second is not. One approach to these concerns is sending the bytes from B to C at CropTarget creation time. And start cropping + store them in a round robin cache to optimise the regular cases. Caching is a well known optimisation tool so I think we can qualify this as a simple approach.

These two mitigations seem honestly simpler than your proposed approach that adds the following wonderful complexity:
- It requires UA to deal with memory caps in a smart manner
- It requires Web Apps to deal with rejection edge cases that can happen out of the web app control (GC, same process navigation, third party iframes...).

To make progress here, I think we should try to focus on https://github.com/w3c/mediacapture-region/issues/48 first.

-- 
GitHub Notification of comment by youennf
Please view or discuss this issue at https://github.com/w3c/mediacapture-region/issues/17#issuecomment-1132604433 using your GitHub account


-- 
Sent via github-notify-ml as configured in https://github.com/w3c/github-notify-ml-config

Received on Friday, 20 May 2022 08:04:43 UTC