Re: [mediacapture-transform] Out-of-main-thread processing by default (#23)

> > * Applications migrating from canvas that have all their logic on the main thread and seek a gradual migration (we've seen this in among some participants in our origin trial)
> 
> My guess is that applications complexity mostly lies in the processing of the data, not in how to get the data: WebGL shaders, WASM libraries...

My guess is that In a complex codebase with a lot of interconnected layers, introducing off-main-thread processing can be a challenging architecture change, and having the possibility of a migration in multiple stages can be very important for such a project. For example, they could first migrate the main-thread logic from canvas to the new API and evaluate the correctness of that change, in a second stage they could better isolate the part of the processing that will move off the main thread, and in a third stage actually they could move the logic off the main thread.
Allowing reasonably easy main-thread usage, even if it's not the default is important in this kind of scenarios, which exist in the real world. 

> I think the requirement is to make sure such processing are feasible and easy to deploy in the environments we are envisioning.
> 
> > * Any application handling low loads including demos and toys, but also some production applications.
> 
> Demos should show the preferred way of using the API. Toys are product applications.
> In general, this is a difficult choice and it highly depends on the application and the device running the application.
> Say a device with a buffer pool of 3 frames, application doing 10fps and GC triggering random hundred millisecond freezes.
> 
> As an example, ScriptProcessorNode is probably fine for some applications on some devices.
> 
> > In fact, it can be a way to shoot yourself in the foot. If you do processing that requires a lot of CPU on a real-time priority thread you run the risk of starving everything else, including UI and other things running on the main thread.
> 
> Agreed and that is precisely the point: isolate potentially perf sensitive tasks in their own context to let the UA do the prioritisation, potentially dynamically.

I presented a proposal to have main-thread processing disabled by default, but to make it easily available if the application explicitly needs it. Do you think it addresses the concern originally raised by this spec issue, even if you disagree with the general approach taken by the spec? 


-- 
GitHub Notification of comment by guidou
Please view or discuss this issue at https://github.com/w3c/mediacapture-transform/issues/23#issuecomment-843104634 using your GitHub account


-- 
Sent via github-notify-ml as configured in https://github.com/w3c/github-notify-ml-config

Received on Tuesday, 18 May 2021 11:53:38 UTC