Re: [mediacapture-transform] A review (#38)

> It seems that we have agreement that processing on main thread should not be impossible.

I would phrase it this way:
- There is no consensus on a goal to provide an API that allows processing on main thread.
- There is consensus on not having a goal to forbid main thread processing.

> It's not clear to me why this would be needed. I asked for further clarification in #27.

I added some comments in the thread.


> The algorithms are not really needed since the spec is defined in terms of the results the APIs

The current spec would be really hard to implement in an interoperable manner.
Some JS exposed behaviours are actually not specified for instance.
Describing these algorithms will allow reaching interoperability.


> Can you elaborate on what is unclear about it? I replied to #24 a while ago and haven't received any extra feedback since them.

I added some comments in the thread.


> Already replied to by Chris in [#38 (comment)](https://github.com/w3c/mediacapture-transform/issues/38#issuecomment-842808300)
> Still unclear?

@aboba might clarify things here. My understanding is as follow:
- We have MediaStreamTrack data that will often be backed by GPU data
- We have WebGPU API that will allow processing GPU data
- Is this API able to do WebGPU processing on MediaStreamTrack data in an optimal manner, ideally no conversion/no memory copies? If not, do we have a good idea on how to add that support at a later stage.


> > * Can one JS document hose a browser's camera capturing ability? #37
> 
> Not by default, but yes, if the document reads enough frames and does not close them a camera capturer can suspend producing frames.

The general question is how much we can reduce this risk and whether APIs could be designed with this in mind.
For instance, the current API is exposing a pattern that seems nice: when enqueueing a frame in a MediaStreamTrack generator, the frame gets consumed.
I would hope we can continue improving the design to further reduce these risks.

-- 
GitHub Notification of comment by youennf
Please view or discuss this issue at https://github.com/w3c/mediacapture-transform/issues/38#issuecomment-852112841 using your GitHub account


-- 
Sent via github-notify-ml as configured in https://github.com/w3c/github-notify-ml-config

Received on Tuesday, 1 June 2021 13:12:32 UTC