Re: [mediacapture-transform] Out-of-main-thread processing by default (#23)

> > The option of using [w3c/mediacapture-extensions#16](https://github.com/w3c/mediacapture-extensions/issues/16) also would allow getting access on the main thread.
> 
> It'd let us limit new MST access methods to workers, so users have to transfer them to a worker to do such processing.
> 

True. But is the any precedent for such an API design?.
In general, I think the approach for APIs is to expose on both the main thread and the worker and let application developers decide based on the requirements of their applications.

> > I don't think allowing access on the main thread is such a dangerous thing that should be forbidden at all costs,
> 
> I think it's [generally understood](https://web.dev/mainthread-work-breakdown/) that the [main-thread is overworked and underpaid](https://www.youtube.com/watch?v=7Rrv9qFMWNM&t=336s). It's a particularly poor environment for realtime (low-latency) processing due to the open-ended number of tasks that may be queued on it at certain times, which can lead to ms delays even to get _on_ it. This is why _"The main-thread is completely unpredictable"_. Newer devices have increasingly tighter FPS deadlines to meet, so we shouldn't design something that cannot meet it reliably.
> 

Yes. And developers know that, so they are in a good position to weigh the tradeoffs. I don't think we should make the decision for them. Moreover, the reality today is that many different types of applications, from simple examples to industrial-strength VC systems are using the main thread today to do this type of processing and it works.


> > especially if the result is a much more complex API
> 
> I don't see how it would be. Worst-case, streams can be transferred from workers to main-thread, no worse than what was proposed the other direction.

I agree that transferring tracks would not result in a particularly complex API. My comment was before we had decided to move in that direction. 


-- 
GitHub Notification of comment by guidou
Please view or discuss this issue at https://github.com/w3c/mediacapture-transform/issues/23#issuecomment-838593804 using your GitHub account


-- 
Sent via github-notify-ml as configured in https://github.com/w3c/github-notify-ml-config

Received on Tuesday, 11 May 2021 14:39:01 UTC