Re: Raw data API - 4 - direct RTP access

My main concern with this API is that we don't have a clear scope on 
what we intend to do on javascript and what should be provided by the 
browser stack.

As Bernard has already mentioned, is FEC/RTX meant to be implemented on 
the js layer on the stack or a mix of both? If you decide to implement 
the FEC on the stack, how do you allow to receive RTCP nacks on the js 
layer or being handled by the stack? Do I want to manually send the RTCP 
NACKs for missing packets or not? The  Same with bandwidth management, 
how do we control/prevent what is the flow of the rtcp messages desired?

The amount of switches and configurations parameters to control all of 
that is just too huge to be usable. IMHO we should target to a "media 
only RTP" api (with no rtcp) or a full RTP/RTCP apis delegating all the 
responsibility to the js layer.

Best regards
Sergio

On 29/05/2018 8:31, Harald Alvestrand wrote:
>
> **
>
>
>   *Low-level RtpTransport*
>
> *
>
> One alternative to extending the RTPSender interface is to define a 
> new RTPTransport interface that takes RTP packets as its input/output 
> format. This would allow applications that desire to do so to 
> implement the entire RTP stack (or a tweaked RTP stack) in Javascript 
> or WebAssembly.
>
>
> Note: before deciding this level, we have to decide whether congestion 
> control and encryption lives above or below the interface. If we do an 
> API that does encryption in user space, we also expose keys to 
> Javascript, which is not always a Good Thing.
>
> The transport has to police the congestion control, so the interface 
> has to take congestion control signals from the transport and take 
> appropriate action in choosing to send or not to send.
>
> This (API for congestion control) is said to be an advantage of the 
> Streams API, so this should at least be looked at.
>
>
> An example interface:
>
>
> // A DtlsTransport and/or IceTransport underneath
>
> // This is encrypted and congestion-controlled
>
> interface RtpTransport {
>
>  sendRtpPacket(RtpPacket packet);
>
>  sendRtcpPacket(RtcpPacket packet);
>
>  attribute eventhandler onrtppacket;
>
>  attribute eventhandler onrtcppacket;
>
> }
>
>
> dictionary RtpPacket {
>
>  // Metadata/"control block"
>
>  unsigned octet payloadType;
>
>  unsigned int sequenceNumber;
>
>  unsigned long ssrc;
>
>  array<RtpHeaderExtension> header_extensions;
>
>  array<unsigned long> csrcs;
>
>
>  bytes payload;
>
> }
>
>
> dictionary RtpHeaderExtension {
>
>  unsigned int id;
>
>  bytes value;
>
> }
>
>
> dictionary RtcpPacket {
>
>  // ...
>
> }
>
>
> A streams interface:
>
>
> Interface RtpTransport {
>
>   ReadableStream<RtpPacket, RtcpPacket> reader;
>
>   WritableStream<RtpPacket, RtcpPacket> writer;
>
> }
>
>
> (note: Syntax for defining acceptable types for a stream invented by hta)
>
> *
> -- 
> Surveillance is pervasive. Go Dark.