Re: [mediacapture-transform] We shouldn't require track transferability (#113)

> would it be OK to support `MediaStreamTrackProcessor` and `VideoTrackGenerator` on the main thread

If the idea is to keep the whole processing in main thread, Chromium `MediaStreamTrackProcessor` and `VideoTrackGenerator` can be shimed with a combination of `rvfc`, `VideoFrame` and `canvas`.

If the app wants to do `VideoFrame` handling out of main thread in an optimal way, the standardized API is the way forward.

> many applications prefer to have the track on Window

It is indeed sad that the transition for these apps is not straightforward, I hope we keep that in mind for future work.

> My claim is that a shim that provides the full functionality and semantics of a track on Window is more complex and has more performance issues

@guidou, I'd like to hear about these `performance issues`, so far I am still very unclear what these issues actually are.

As of `more complex`, a large part of the complexity may actually come from the fact that these apps currently have to deal with both Chrome proprietary API and the standard API.
It would probably be simpler if the standard API was supported in all browsers.

> If the track is transferred, we would have to maintain duplicate versions of this state

I am not sure about this duplication.
I agree though that, to keep the whole adaptation logic in window, the worker would need to send back the `applyConstraints` results back to window.

-- 
GitHub Notification of comment by youennf
Please view or discuss this issue at https://github.com/w3c/mediacapture-transform/issues/113#issuecomment-2589283776 using your GitHub account


-- 
Sent via github-notify-ml as configured in https://github.com/w3c/github-notify-ml-config

Received on Tuesday, 14 January 2025 08:19:52 UTC