Re: [webrtc-encoded-transform] Expose RTCEncoded*Frame interfaces in Worklets (#226)

> WebRTC encoded transform streams are transferable but RTCRtpScriptTransformer is not.
The RTCRtpScriptTransformer.readable is transferable, so could be posted to a worklet within the current shape.

The former usecase @youennf mentioned - decoding+rendering Audio - is indeed the one I'm interested in getting on a worklet. IIUC libwebrtc does its audio decoding in the realtime thread, just before rendering, so that concept isn't all that wild.

> Any claim of being able to do a "more performant" implementation need to be backed by something.
Transferring the readablestream to the worklet would mean frames could be delivered directly there. Requiring JS work to be done elsewhere first would necessitate visiting another JS thread, scheduling an event there etc, so ~double the overhead plus the cost of allocating the intermediate objects to be re-transferred. I can see if I can get some more concrete numbers, but there's clearly additional work needed to be done by app+UA which could be skipped with this.

-- 
GitHub Notification of comment by tonyherre
Please view or discuss this issue at https://github.com/w3c/webrtc-encoded-transform/issues/226#issuecomment-1985612081 using your GitHub account


-- 
Sent via github-notify-ml as configured in https://github.com/w3c/github-notify-ml-config

Received on Friday, 8 March 2024 12:31:09 UTC