Re: [EXTERNAL] Call for Consensus (CfC) on Adoption of the Insertable Streams API specification

Following on yesterday interim meeting, it seems there is consensus that e2e encryption is of great interest to people interested in insertable streams.
We also want in a not-too-far future that web sites will not have to do their own crypto to implement it.

Based on that, I think we should make a requirement that insertable streams be able to support native nodes/transforms, at least one dedicated to encrypting media.
In parallel to working on insertable streams, or as part of it, we should actually define such a transform, probably based on the SFrame proposal.
We can hopefully isolate key management like HTMLMediaElement did with EME.

I see several advantages:
- We can ship features progressively with good confidence that we will not get stuck in a JS-only solution
- We lay down a foundation where insertable streams is reusable with a key management system built in the browser (MLS?)
- By specifying this specific transform, we will define hooks in the insertable stream spec required for this transform, some of these hooks surfacing at the JS level for proper polyfilling

> On 3 Jun 2020, at 07:52, Stefan Håkansson LK <stefan.lk.hakansson@ericsson.com> wrote:
> 
> Thanks Bernard, sounds good. So a follow question could be: couldInsertable Streams somehow provide handles to GPU buffers which could enable WebGPU operations? Just thinking out loud (I should go and do some investigation myself).
>  
> From: Bernard Aboba <Bernard.Aboba@microsoft.com>
> Date: Tuesday, 2 June 2020 at 19:58
> To: Stefan LK <stefan.lk.hakansson@ericsson.com>
> Cc: "public-webrtc@w3.org" <public-webrtc@w3.org>
> Subject: Re: [EXTERNAL] Re: Call for Consensus (CfC) on Adoption of the Insertable Streams API specification
>  
> On Jun 2, 2020, at 2:00 AM, Stefan Håkansson LK <stefan.lk.hakansson@ericsson.com> wrote:
>>  
>> I’d like to see more work to determine if this model is also suitable for accessing raw media, and I’d also like to see overlap/sync vis a vis WebCodecs investigated.
>> Regarding N22: would it be worth looking into whether WebGPU (https://gpuweb.github.io/gpuweb/ <https://protect2.fireeye.com/v1/url?k=8f90bc3a-d1307e7c-8f90fca1-86fc6812c361-7251295517553986&q=1&e=daa517f5-5c14-48d2-a604-2f6cad31d9ac&u=https%3A%2F%2Fnam06.safelinks.protection.outlook.com%2F%3Furl%3Dhttps%253A%252F%252Fgpuweb.github.io%252Fgpuweb%252F%26data%3D02%257C01%257CBernard.Aboba%2540microsoft.com%257C8db5a4fef3a94a7679e408d806d32991%257C72f988bf86f141af91ab2d7cd011db47%257C0%257C0%257C637266852299530210%26sdata%3D%252BaA%252FeU2LsVhlpAjFBioVPdLOdsoygrQ3yzN8yo0ui6w%253D%26reserved%3D0>) could be useful?
>  
> [BA] Excellent points. Raw media is much larger than encoded, so performance is a concern. One thing under discussion in WebCodecs is whether it might be useful to provide handles to GPU buffers which could enable WebGPU operations on raw (or encoded) media.
>  
> Prototyping of WHATWG stream support in WebCodecs did not go well, and issues relating to WebRTC parity and DRM support have been raised. So at present Insertable Streams is much closer to a “bird in the hand” although we are not sure what songs this bird may be able to sing.