- From: Jan-Ivar Bruaroey via GitHub <sysbot+gh@w3.org>
- Date: Mon, 02 Aug 2021 20:00:01 +0000
- To: public-webrtc@w3.org
jan-ivar has just created a new issue for https://github.com/w3c/mediacapture-transform: == MSTP's VideoFrame lifetime management model assumes a single consumer == `MediaStreamTrackProcessor` [appears to expect](https://w3c.github.io/mediacapture-transform/#video-processing) _consumers_ to call `VideoFrame.close()` manually. Unfortunately, this: 1. Assumes a single consumer per VideoFrame 2. Creates a footgun where consumers may forget to call this method, with potentially devastating OS-level side-effects Ditto `MediaStreamTrackGenerator` which [calls](https://w3c.github.io/mediacapture-transform/#generator) `VideoFrame.close()` implicitly, preventing reuse with another sink. It too: 1. Assumes a single consumer per VideoFrame `VideoFrame`s are immutable objects (with a necessary mutable `close()` method because we don't want to wait for GC, a pattern that works well for a single owner). A benefit of immutability is supposed to be shareability, so we should be able to tee a stream: ```js const [readable1, readable2] = new MediaStreamTrackProcessor({track}).readable.tee(); await Promise.all([ readable1.pipeThrough(new MyTransform(640)).pipeTo(sink1), readable2.pipeThrough(new MyTransform(320)).pipeTo(sink2) ]); ``` ...and this would work, except it breaks if `MyTransform` calls `VideoFrame.close()`. IOW, since sources may have multiple sinks, it seems better for sources to own the `VideoFrame`s it doles out, and manage their lifetime. ``` source -.---> sink1 `---> sink2 ``` ### Proposal * Have `MediaStreamTrackGenerator` manage lifetime of `VideoFrame`s it passes to immediate sinks. * _(and any transform stream that wants to pass through a `VideoFrame` needs to copy or renew it somehow)_ * Stop having `MediaStreamTrackGenerator` call `VideoFrame.close()` and rely on the producer to do this. Please view or discuss this issue at https://github.com/w3c/mediacapture-transform/issues/56 using your GitHub account -- Sent via github-notify-ml as configured in https://github.com/w3c/github-notify-ml-config
Received on Monday, 2 August 2021 20:00:03 UTC