Re: Raw data API - 3 - Encoded data

I fail to see what would be the benefits of adding the media frame API 
to the RTP objects, especially if we intend to provide a lower raw rtp API.

In order to be able to implement this API on the browser, the codec 
packetization of the encoded stream must be known by the browser. Also, 
it is not possible to modify the frame, for example applying end to end 
media encryption, as packetization requires to have access to the raw 
data or add metadata for the very same reason.

If the API would only alow to forward the frames from an encoder/decoder 
(or another media source/sink) to the rtp objects, I would prefer a 
higher level API that deals with streams and not individual packets.

Best regards
Sergio

On 29/05/2018 8:30, Harald Alvestrand wrote:
>
> **
>
>
>     *Access to encoded streams*
>
> *
>
> A similar interface can be added to RTPSender and RTPReceiver, 
> respectively (and similar APIs for other transports, when defined).
>
>
> Here the buffers would contain encoded video / audio data, and the 
> control block parameters would have to have enough information to let 
> the RTP packet headers be constructed - or, on the receiver, the info 
> from the RTP packet headers be represented.
>
>
> Partial interface RTPSender {
>
>    promise<encodedBuffer> injectData(encodedBuffer);
>
> }
>
>
> Partial interface RTPReceiver {
>
>    promise<encodedBuffer> extractData(encodedBuffer);
>
> }
>
>
> Interface encodedBuffer : Buffer {
>
>    Long rtpTimestamp;
>
>    Long frameId;
>
>    sequence<long> dependsOnFrames;
>
>     // more fields TBD
>
> }
>
>
> On this interface, frames do have interdependencies, so dropping 
> packets is much more problematic. The “dependsOnFrames” member is 
> intended to help deciding on sensible handling - it would tell the 
> other side of the API that “if you dropped one of these frames, you 
> might as well drop this frame too”.
>
> *