Re: [EXTERNAL] Re: Call for Consensus (CfC) on Adoption of the Insertable Streams API specification

Thanks Bernard, sounds good. So a follow question could be: could Insertable Streams somehow provide handles to GPU buffers which could enable WebGPU operations? Just thinking out loud (I should go and do some investigation myself).

From: Bernard Aboba <Bernard.Aboba@microsoft.com>
Date: Tuesday, 2 June 2020 at 19:58
To: Stefan LK <stefan.lk.hakansson@ericsson.com>
Cc: "public-webrtc@w3.org" <public-webrtc@w3.org>
Subject: Re: [EXTERNAL] Re: Call for Consensus (CfC) on Adoption of the Insertable Streams API specification

On Jun 2, 2020, at 2:00 AM, Stefan Håkansson LK <stefan.lk.hakansson@ericsson.com> wrote:

I’d like to see more work to determine if this model is also suitable for accessing raw media, and I’d also like to see overlap/sync vis a vis WebCodecs investigated.
Regarding N22: would it be worth looking into whether WebGPU (https://gpuweb.github.io/gpuweb/<https://protect2.fireeye.com/v1/url?k=8f90bc3a-d1307e7c-8f90fca1-86fc6812c361-7251295517553986&q=1&e=daa517f5-5c14-48d2-a604-2f6cad31d9ac&u=https%3A%2F%2Fnam06.safelinks.protection.outlook.com%2F%3Furl%3Dhttps%253A%252F%252Fgpuweb.github.io%252Fgpuweb%252F%26data%3D02%257C01%257CBernard.Aboba%2540microsoft.com%257C8db5a4fef3a94a7679e408d806d32991%257C72f988bf86f141af91ab2d7cd011db47%257C0%257C0%257C637266852299530210%26sdata%3D%252BaA%252FeU2LsVhlpAjFBioVPdLOdsoygrQ3yzN8yo0ui6w%253D%26reserved%3D0>) could be useful?

[BA] Excellent points. Raw media is much larger than encoded, so performance is a concern. One thing under discussion in WebCodecs is whether it might be useful to provide handles to GPU buffers which could enable WebGPU operations on raw (or encoded) media.

Prototyping of WHATWG stream support in WebCodecs did not go well, and issues relating to WebRTC parity and DRM support have been raised. So at present Insertable Streams is much closer to a “bird in the hand” although we are not sure what songs this bird may be able to sing.