[mediacapture-insertable-streams] Is using ReadableStream/WritableStream the right approach? (#4)

youennf has just created a new issue for https://github.com/w3c/mediacapture-insertable-streams:

== Is using ReadableStream/WritableStream the right approach? ==
The current proposed API is based on ReadableStream of frames.
It does not seem that pros and cons of this approach have been documented.
It would also be interesting to list what other approaches could be envisioned and evaluate them.

For WebRTC insertable streams, we anticipate defining native transforms so it is convenient to be able to use streams algorithms like pipeTo/pipeThrough for free. And there is no stream-like object we can hook to.

If we look at media capture, we already have the MediaStreamTrack which is a kind of ReadableStream.
For instance, it can piped into web audio or generated from web audio or canvas.
It can also be cloned.
If we want to have a native transform that does WebGL stuff, we could simply have a method that takes a MediaStreamTrack, some WebGL stuff and create a new MediaStreamTrack.

Downsides mentioned for webrtc insertable streams can also be mentioned here.
For instance, cloning a ReadableStream is not as safe as cloning a MediaStreamTrack.
Or the fact that this API makes it convenient to do processing in the main thread, which might be a real foot gun.

Please view or discuss this issue at https://github.com/w3c/mediacapture-insertable-streams/issues/4 using your GitHub account


-- 
Sent via github-notify-ml as configured in https://github.com/w3c/github-notify-ml-config

Received on Tuesday, 1 December 2020 14:38:17 UTC