Re: [mediacapture-transform] A review (#38)

> > It seems that we have agreement that processing on main thread should not be impossible.
> 
> I would phrase it this way:
> 
> * There is no consensus on a goal to provide an API that allows processing on main thread.
> * There is consensus on not having a goal to forbid main thread processing.

I agree with this phrasing.


> 
> The current spec would be really hard to implement in an interoperable manner.

Are you referring to the streams algorithms or something else specifically?


> Some JS exposed behaviours are actually not specified for instance.

Can you file specific bugs about those behaviors?


> Describing these algorithms will allow reaching interoperability.

I'll prepare a PR to include those algorithms. Tracked on #36.


> * We have MediaStreamTrack data that will often be backed by GPU data
> * We have WebGPU API that will allow processing GPU data
> * Is this API able to do WebGPU processing on MediaStreamTrack data in an optimal manner, ideally no conversion/no memory copies? If not, do we have a good idea on how to add that support at a later stage.

I think we should continue this discussion on a separate bug, perhaps on the WebCodecs spec if we agree that WebCodec's VideoFrame is the right API to expose video frames coming from a MediaStreamTrack.

> 
> > > * Can one JS document hose a browser's camera capturing ability? #37
> > 
> > 
> > Not by default, but yes, if the document reads enough frames and does not close them a camera capturer can suspend producing frames.
> 
> The general question is how much we can reduce this risk and whether APIs could be designed with this in mind.
> For instance, the current API is exposing a pattern that seems nice: when enqueueing a frame in a MediaStreamTrack generator, the frame gets consumed.
> I would hope we can continue improving the design to further reduce these risks.

I agree that the goal should be risk reduction since completely eliminating this possibility would negate the goal of optimal WebGPU processing.


-- 
GitHub Notification of comment by guidou
Please view or discuss this issue at https://github.com/w3c/mediacapture-transform/issues/38#issuecomment-852236943 using your GitHub account


-- 
Sent via github-notify-ml as configured in https://github.com/w3c/github-notify-ml-config

Received on Tuesday, 1 June 2021 15:52:46 UTC