Re: [webrtc-encoded-transform] Should we expose packetization level API to RTCRtpScriptTransform? (#131)

We would need the following info for exposing the RTP packetization for each frame:

```

dictionary RTCRTPFragment{
    attribute Buffer prefix;
    attribute ArrayBufferView payload; //Containing the correct fragment of the video frame payload
    attribute Buffer suffix;
}

partial interface RTCEncodedVideoFrame {
    attribute sequence<RTCRTPFragment> payload;
}
```

Note that the RTP packetization may require to prepend some RTP header stuff in each packet (nal headers in h264, vp8 picture headers) and may skip some data on the payload (the nal headers in h264, for example). I am not aware of any packetization that requires appending data at the end, but may be worthy.

Encrypting could be straight forward, as it could just put all the encrypted buffer as the `prefix` or `sufix` and skip the `payload` completely. 

What is a bit more complex is in decryption, because as it is happening *before* the depacketization, we don't have a assembled video frame, and we would need to pass the raw RTPpacket instead (either just the payloads, or the full rtp packet info)




-- 
GitHub Notification of comment by murillo128
Please view or discuss this issue at https://github.com/w3c/webrtc-encoded-transform/issues/131#issuecomment-1029912573 using your GitHub account


-- 
Sent via github-notify-ml as configured in https://github.com/w3c/github-notify-ml-config

Received on Friday, 4 February 2022 11:43:11 UTC