Re: [mediacapture-transform] Out-of-main-thread processing by default (#23)

> > I'm saying that going against the established pattern of exposing in both places requires a stronger argument than "the user might do something wrong in certain circumstances" which is the only argument given so far.
> 
> I'm hearing no reason other than pattern, and I suspect those patterns are largely historical. I.e. workers are still a relatively young concept compared to the size of the platform, so that might explain why many APIs were exposed to main thread first. WebIDL lets us specify exposure discretely, and I see no established _rationale_ to back up why exposure must follow any pattern. If you have one, please point me to it, otherwise I see no reason to follow a pattern not backed by rationale.
> 

The argument is that there are use cases where allowing the main thread is justified IMO:
- Applications migrating from canvas that have all their logic on the main thread and seek a gradual migration (we've seen this in among some participants in our origin trial)
- Any application handling low loads including demos and toys, but also some production applications.

I don't think ignoring these legitimate use cases because the main thread is bad in other situations is enough justification.  

> Also, AudioWorklet is a clear example that breaks that pattern: we had main thread exposure, and removed it on purpose for good reasons, which I [gave](https://github.com/w3c/mediacapture-transform/issues/23#issuecomment-836812770). That this ended up being a worklet rather than a worker, I don't think had anything to do with any rule that APIs cannot be exposed in workers only. I've also seen no proposals to transfer `ScriptProcessorNode`.
> 
> I also disagree that being conservative requires arguments. I think being cautious with these low-level APIs is the right approach, which means requiring arguments to be anything but conservative. It'll be hard to put the genie back in the bottle.
> 

My point is that there are some legitimate use cases that we shouldn't ignore and the historical precedent backs supporting those use cases.  Now, what is actually the issue? the concern originally raised by this spec issue is main thread by default, but we're mainly discussing allowing or not allowing running on the main thread, which is a different discussion.

We can address the concern of main thread by default with an extra parameter in the constructor (e.g., allowStreamOnWindow) that is false by default. Any attempt to use the streams on Window would fail unless the application explicitly passes this parameter as true, therefore disabling processing on main thread by default. Would this address the concern in your opinion? 


> > An advantage worklets can provide is the ability to run on any thread, including real-time threads, but that is not needed for our intended use cases.
> 
> What use case is that? "[Funny hats](https://w3c.github.io/mediacapture-transform/#introduction)" and "Virtual Reality Gaming" both seem real-time to me.

The fact that it is a "real-time" application does not mean a real-time priority thread at the OS level is a good idea. In fact, it can be a way to shoot yourself in the foot. If you do processing that requires a lot of CPU on a real-time priority thread you run the risk of starving everything else, including UI and other things running on the main thread. 


-- 
GitHub Notification of comment by guidou
Please view or discuss this issue at https://github.com/w3c/mediacapture-transform/issues/23#issuecomment-843003062 using your GitHub account


-- 
Sent via github-notify-ml as configured in https://github.com/w3c/github-notify-ml-config

Received on Tuesday, 18 May 2021 09:16:12 UTC