Transport / encoder coupling (Re: Use cases / requirements for raw data access functions)

On 05/19/2018 12:11 AM, Sergio Garcia Murillo wrote:
>>
>> We should separate transports from encoders (split the RtpSender in
>> half) to give more flexibility to apps.
>
> I agree with that, but given RTP packetization is codec specific, we
> can't ignore that.

I think we can translate this into requirements language:

IF we specify the transport separately from the encoder

THEN we need a number of pieces of API, either as JS API or as
specifications of "what happens when you couple a transport with an
encoder":

- If the transport is like RTP, and isn't codec agnostic, it needs
information from the encoder about what the codec is, and may need to
inform the encoder about some transport characteristics (like max MTU
size for H.264 packetization mode 0).

- If the transport is congestion controlled (and they all are!), then
there needs to be coupling of the transport's congestion control with
the encoder's target bitrate

- If the transport isn't reliable, and may toss frames, it needs to be
coupled back to the encoder to tell it to do things like stop referring
to a lost frame, or to go back and emit a new iframe.

Handling the data is just one part of handling the interaction between
transport and encoder.

-- 
Surveillance is pervasive. Go Dark.