Re: [webrtc-encoded-transform] Encoded frame IDLs share definitions with WebCodecs (#99)

HI @youennf, sorry for the delay. 

Encoded*Chunk currently exposes a readonly data attribute, but we were planning to remove it entirely in favor of a readInto() method that copies the data out ([PR for this here](https://github.com/w3c/webcodecs/pull/174)). The reasoning is that we wish for all I/O types in webcodecs to be immutable to [avoid surprising users](https://github.com/w3c/webcodecs/issues/80) and (critically) to [avoid TOCTOU security issues](https://github.com/w3c/webcodecs/issues/127#issuecomment-770054107). 

I saw your [comment here](https://github.com/w3c/webrtc-encoded-transform/pull/101/files#r613989234)
> Now that I look at it, data is readonly here but we want to be able to change it in RTCRtpSFrameTransform transforms.

Rather than setting the .data attribute, is it possible to instead construct a new chunk w/ the transformed data? We could design a copy constructor or similar to make this easy for authors. 

> Define the additional metadata fields using partial dictionaries

We've had some other requests for metadata as well. I'd like to weigh the options of subclassing, vs wrapping, vs extending (with partial dictionaries) vs having an open ended metadata bucket (maybe a Map) on the Encoded*Chunk interfaces. I don't have an opinion just yet. 

For the metadata you're seeking, is it important that the metadata be carried through decoding/encoding, such that a VideoFrame corresponding to an EncodedVideoChunk has all the same values?

> For instance, we probably want to stick with readonly timestamps.

No objection. 

-- 
GitHub Notification of comment by chcunningham
Please view or discuss this issue at https://github.com/w3c/webrtc-encoded-transform/issues/99#issuecomment-822879613 using your GitHub account


-- 
Sent via github-notify-ml as configured in https://github.com/w3c/github-notify-ml-config

Received on Tuesday, 20 April 2021 00:24:29 UTC