Re: Use cases / requirements for raw data access functions

It sounds like what you're saying is that in our quest to make lower-level
APIs, you don't want it to go so low that it's a pain in the neck to use.
Which I can completely understand.  Finding the right balance between easy
to use and flexible and powerful  can be tricky, and we should keep your
viewpoint (the "keep it easy" one) in mind as we consider the tradeoffs in
design.

But I think that's mostly orthogonal to QUIC vs. RTP.

On Fri, May 18, 2018 at 3:10 PM Sergio Garcia Murillo <
sergio.garcia.murillo@gmail.com> wrote:

> On 18/05/2018 23:33, Peter Thatcher wrote:
>
> On Fri, May 18, 2018 at 1:28 AM Sergio Garcia Murillo <
> sergio.garcia.murillo@gmail.com> wrote:
>
>> IMHO we (me first) are not providing use cases, but
>> features/functionalities we like to bring into the API, which is fine, but
>> I think we are overlooking one fact: QUIC and RTP have one fundamental
>> difference, udp fragmentation/packetization.
>>
>> While in QUIC the packetization is codec agnostic and performed deep into
>> the stack, in RTP, the packetization is codec dependent and done before
>> reaching the rtp stack.
>>
>> Why I am bringing that topic in? Because I feel that some use
>> cases/features, for example raw access to encoded frames or the bring your
>> own crypto makes a lot of sense for QUIC (where you just need to pass the
>> raw binary data as a whole) but much less sense for RTP.
>>
>
> It makes just as much sense for RTP.  Having access to encoded frames
> before RTP packetization is useful for e2ee for RTP as well.  And if the
> RTP transport API is low-level enough, it would be just as easy to add
> arbitrary metadata (another use case mentioned) as it is for QUIC.
>
>
> Can't disagree more.
>
>
>
> Don't take me wrong, I am not against that use cases at all (I like them),
>> but as I see it, we should consider QUIC as an alternative to DataChannels
>> from an API/use case point of view, and not as an alternative to RTP.
>>
>
> QUIC is a transport, not a replacement for RTP.  But you can build a
> replacement for RTP on top of QUIC (or an top of SCTP, for that matter).
> Just as you could make an RTP data channel and build a replacement for RTP
> on top of RTP.
>
>
> Except that I don't want to implement RTP on top of QUIC, I just want to
> use it as is.
>
>
>
> We should separate transports from encoders (split the RtpSender in half)
> to give more flexibility to apps.
>
>
> I agree with that, but given RTP packetization is codec specific, we can't
> ignore that.
>
>
> My worries are that we try to create an API that cover those use cases
>> that works for both QUIC and RTP, which will create an awful experience for
>> those willing to use RTP, or even worse, not even consider the RTP
>> specifics at all (as you already have the "raw" functionality and you can
>> implement that on your own on your javascript app), and RTP becoming a
>> second class citizen on webrtc.
>>
>
> How would splitting an RtpSender into an encoder and transport be an awful
> experience?  You can do everything you can currently do, just more.
>
>
> If I need to get the encoded frame, do manually the packetization, add
> headers, handle buffers, etc.. then it is ugly, if it is "just"
> encoder.sendTo(sender) (or whatever similar in complexity), then it is not
> ugly.
>
>
> I don't think this is choice between RTP, QUIC, and SCTP.  We can make
> them all work.    But the first step is decoupling encoder/decoders,
> transports, and ICE.  Then apps/developers can assemble the parts they want
> the way they want.
>
> But, as you say, let's start with use cases, not solutions.
>
> By the way, this is my use case: as a web/mobile developer, I want to send
> media over QUIC for my own replacement for RTP.  QUIC isn't the replacement
> for RTP, but my own protocol on top of QUIC (or SCTP, or any data channel)
> is the replacement for RTP.    Luckily, this is "free" as soon as you add
> QUIC data channels and split the RtpSender into encoder and transport.
> That's all I need.
>
> That's not an use case, but a technical implementation detail, anyway, I
> see its value and support it, as long as it is not the only way to do it
> and I am not forced to write tons of code just to do the same on RTP.
> That's all I am asking for.
>
> Best regards
>
> Sergio
>
>
>