Re: [mediacapture-transform] A review (#38)

> * Don’t expose raw realtime media on main thread, since there's a precedent that says this is bad [#23 (comment)](https://github.com/w3c/mediacapture-transform/issues/23#issuecomment-842585817)
This has been replied to in #23. It seems that we have agreement that processing on main thread should not be impossible.
We still do not have consensus on how hard it should be (no one has replied to my proposal to have it off by default and require an explicit constructor parameter to enable it). 
There are valid use cases for using the main thread. For example, easier migration from canvas capture and any application where not a lot of processing occurs and using a Worker would add no benefit other than some extra resource consumption and making the application more difficult to write.

> * Limit to solving video for now, or explain why AudioWorklet is insufficient, to not reinvent it #29
This has been responded to in several messages of #29, but here's a summary:
* Makes it easier to develop applications that need to combine audio and video processing (e.g., https://arxiv.org/pdf/1804.03619.pdf)
* Same API for audio and video, which makes things easier for developers especially if the processing is similar or the same
* Different processing model from audio worklet. Access to direct data from the track, including its timestamps.

> * Lack of a threading model #27
It's not clear to me why this would be needed. I asked for further clarification in #27.

> * Lack of algorithms #36
The algorithms are not really needed since the spec is defined in terms of the results the APIs should produce instead of the steps required to produce them. However, since the streams API recommends using them, I'll prepare a PR to use them (without any change in semantics).

> * Controlling channel mechanism is unclear #24 
Can you elaborate on what is unclear about it? I replied to #24 a while ago and haven't received any extra feedback since them.

> * Unclear memory I/O design given that a VideoFrame may be in GPU memory #34 (Funny hat: GPU→CPU→GPU?)
Already replied to by Chris in https://github.com/w3c/mediacapture-transform/issues/38#issuecomment-842808300
Still unclear?

> * Can one JS document hose a browser's camera capturing ability? #37
Not by default, but yes, if the document reads enough frames and does not close them a camera capturer can suspend producing frames. 


-- 
GitHub Notification of comment by guidou
Please view or discuss this issue at https://github.com/w3c/mediacapture-transform/issues/38#issuecomment-851571074 using your GitHub account


-- 
Sent via github-notify-ml as configured in https://github.com/w3c/github-notify-ml-config

Received on Monday, 31 May 2021 15:50:48 UTC