Re: Raw data API - 4 - direct RTP access

What about RTCP SR/RR/XR? I think that should also be handled by the 
core as well. And NACks, I can't imagine why an app would want to 
deal/mangle with them.

Also, (Flex) FEC should be quite integrated with congestion control as 
it should be the preferred way of bandwidth probing, it will be very 
difficult to make them work together if one is implemented in the core 
and the other in js/wasm.

IMHO, everything that is standard, should be implemented by the core and 
not allow the app to play with it.

Best regards
Sergio


On 30/05/2018 18:32, Peter Thatcher wrote:
> I think there are many levels of RTP transport we can consider, and we 
> should consider the lowest reasonable level as well, in which the 
> browser would provide:
>
> 1.  Packet serialization (from struct to packet)
> 2.  Congestion control (perhaps by adding its own header extensions or 
> RTCP messages)
> 3.  Crypto (SRTP)
>
> And that's it.
>
> I don't think it's worth considering anything without crypto or 
> congestion control.
>
> Higher-level transport could also be considered, but I think this 
> low-level one would be a good one.
>
> On Tue, May 29, 2018 at 4:08 AM Sergio Garcia Murillo 
> <sergio.garcia.murillo@gmail.com 
> <mailto:sergio.garcia.murillo@gmail.com>> wrote:
>
>     My main concern with this API is that we don't have a clear scope
>     on what we intend to do on javascript and what should be provided
>     by the browser stack.
>
>     As Bernard has already mentioned, is FEC/RTX meant to be
>     implemented on the js layer on the stack or a mix of both? If you
>     decide to implement the FEC on the stack, how do you allow to
>     receive RTCP nacks on the js layer or being handled by the stack?
>     Do I want to manually send the RTCP NACKs for missing packets or
>     not? The  Same with bandwidth management, how do we
>     control/prevent what is the flow of the rtcp messages desired?
>
>     The amount of switches and configurations parameters to control
>     all of that is just too huge to be usable. IMHO we should target
>     to a "media only RTP" api (with no rtcp) or a full RTP/RTCP apis
>     delegating all the responsibility to the js layer.
>
>     Best regards
>
>     Sergio
>
>     On 29/05/2018 8:31, Harald Alvestrand wrote:
>>
>>     **
>>
>>
>>       *Low-level RtpTransport*
>>
>>     *
>>
>>     One alternative to extending the RTPSender interface is to define
>>     a new RTPTransport interface that takes RTP packets as its
>>     input/output format. This would allow applications that desire to
>>     do so to implement the entire RTP stack (or a tweaked RTP stack)
>>     in Javascript or WebAssembly.
>>
>>
>>     Note: before deciding this level, we have to decide whether
>>     congestion control and encryption lives above or below the
>>     interface. If we do an API that does encryption in user space, we
>>     also expose keys to Javascript, which is not always a Good Thing.
>>
>>     The transport has to police the congestion control, so the
>>     interface has to take congestion control signals from the
>>     transport and take appropriate action in choosing to send or not
>>     to send.
>>
>>     This (API for congestion control) is said to be an advantage of
>>     the Streams API, so this should at least be looked at.
>>
>>
>>     An example interface:
>>
>>
>>     // A DtlsTransport and/or IceTransport underneath
>>
>>     // This is encrypted and congestion-controlled
>>
>>     interface RtpTransport {
>>
>>      sendRtpPacket(RtpPacket packet);
>>
>>      sendRtcpPacket(RtcpPacket packet);
>>
>>      attribute eventhandler onrtppacket;
>>
>>      attribute eventhandler onrtcppacket;
>>
>>     }
>>
>>
>>     dictionary RtpPacket {
>>
>>      // Metadata/"control block"
>>
>>      unsigned octet payloadType;
>>
>>      unsigned int sequenceNumber;
>>
>>      unsigned long ssrc;
>>
>>      array<RtpHeaderExtension> header_extensions;
>>
>>      array<unsigned long> csrcs;
>>
>>
>>      bytes payload;
>>
>>     }
>>
>>
>>     dictionary RtpHeaderExtension {
>>
>>      unsigned int id;
>>
>>      bytes value;
>>
>>     }
>>
>>
>>     dictionary RtcpPacket {
>>
>>      // ...
>>
>>     }
>>
>>
>>     A streams interface:
>>
>>
>>     Interface RtpTransport {
>>
>>       ReadableStream<RtpPacket, RtcpPacket> reader;
>>
>>       WritableStream<RtpPacket, RtcpPacket> writer;
>>
>>     }
>>
>>
>>     (note: Syntax for defining acceptable types for a stream invented
>>     by hta)
>>
>>     *
>>     -- 
>>     Surveillance is pervasive. Go Dark.
>
>