Re: Raw data API - 4 - direct RTP access

Den 30. mai 2018 18:44, skrev Sergio Garcia Murillo:
> 
> What about RTCP SR/RR/XR? I think that should also be handled by the
> core as well. And NACks, I can't imagine why an app would want to
> deal/mangle with them.

I can easily imagine people wanting to supply their own NACK handling;
deciding what to do with a NACK involves a tradeoff between retransmit,
new-keyframe, modifying the dependency chain to only depend on
wholly-received frames, and hanging up the call at excessive NACK.

Sadly (based on Internet-telephony experience), I think a lot of people
will say "we'll deal with SR/RR/XR if we really need them" and fail to
get them into production in time - that argues for at least policing
that correct SRs and RRs are sent.

Both of these are "if the underlying layer is RTP", of course.

> 
> Also, (Flex) FEC should be quite integrated with congestion control as
> it should be the preferred way of bandwidth probing, it will be very
> difficult to make them work together if one is implemented in the core
> and the other in js/wasm.

Bandwidth probing doesn't have a spec at the moment, FlexFEC is only one
possibility. I don't think we should try to ossify things that aren't
working well yet.

> IMHO, everything that is standard, should be implemented by the core and
> not allow the app to play with it.

That was the idea that led to TCP being an OS kernel function.
The current popularity of UDP-based protocols for evolving the transport
protocol space is a reaction against that idea.

> 
> Best regards
> Sergio
> 
> 
> On 30/05/2018 18:32, Peter Thatcher wrote:
>> I think there are many levels of RTP transport we can consider, and we
>> should consider the lowest reasonable level as well, in which the
>> browser would provide:
>>
>> 1.  Packet serialization (from struct to packet)
>> 2.  Congestion control (perhaps by adding its own header extensions or
>> RTCP messages)
>> 3.  Crypto (SRTP)
>>
>> And that's it.  
>>
>> I don't think it's worth considering anything without crypto or
>> congestion control.  
>>
>> Higher-level transport could also be considered, but I think this
>> low-level one would be a good one.
>>
>> On Tue, May 29, 2018 at 4:08 AM Sergio Garcia Murillo
>> <sergio.garcia.murillo@gmail.com
>> <mailto:sergio.garcia.murillo@gmail.com>> wrote:
>>
>>     My main concern with this API is that we don't have a clear scope
>>     on what we intend to do on javascript and what should be provided
>>     by the browser stack.
>>
>>     As Bernard has already mentioned, is FEC/RTX meant to be
>>     implemented on the js layer on the stack or a mix of both? If you
>>     decide to implement the FEC on the stack, how do you allow to
>>     receive RTCP nacks on the js layer or being handled by the stack?
>>     Do I want to manually send the RTCP NACKs for missing packets or
>>     not? The  Same with bandwidth management, how do we
>>     control/prevent what is the flow of the rtcp messages desired?
>>
>>     The amount of switches and configurations parameters to control
>>     all of that is just too huge to be usable. IMHO we should target
>>     to a "media only RTP" api (with no rtcp) or a full RTP/RTCP apis
>>     delegating all the responsibility to the js layer.
>>
>>     Best regards
>>
>>     Sergio
>>
>>     On 29/05/2018 8:31, Harald Alvestrand wrote:
>>>
>>>     **
>>>
>>>
>>>       *Low-level RtpTransport*
>>>
>>>     *
>>>
>>>     One alternative to extending the RTPSender interface is to define
>>>     a new RTPTransport interface that takes RTP packets as its
>>>     input/output format. This would allow applications that desire to
>>>     do so to implement the entire RTP stack (or a tweaked RTP stack)
>>>     in Javascript or WebAssembly.
>>>
>>>
>>>     Note: before deciding this level, we have to decide whether
>>>     congestion control and encryption lives above or below the
>>>     interface. If we do an API that does encryption in user space, we
>>>     also expose keys to Javascript, which is not always a Good Thing.
>>>
>>>     The transport has to police the congestion control, so the
>>>     interface has to take congestion control signals from the
>>>     transport and take appropriate action in choosing to send or not
>>>     to send.
>>>
>>>     This (API for congestion control) is said to be an advantage of
>>>     the Streams API, so this should at least be looked at.
>>>
>>>
>>>     An example interface:
>>>
>>>
>>>     // A DtlsTransport and/or IceTransport underneath
>>>
>>>     // This is encrypted and congestion-controlled
>>>
>>>     interface RtpTransport {
>>>
>>>      sendRtpPacket(RtpPacket packet);
>>>
>>>      sendRtcpPacket(RtcpPacket packet);
>>>
>>>      attribute eventhandler onrtppacket;
>>>
>>>      attribute eventhandler onrtcppacket;
>>>
>>>     }
>>>
>>>
>>>     dictionary RtpPacket {
>>>
>>>      // Metadata/"control block"
>>>
>>>      unsigned octet payloadType;
>>>
>>>      unsigned int sequenceNumber;
>>>
>>>      unsigned long ssrc;
>>>
>>>      array<RtpHeaderExtension> header_extensions;
>>>
>>>      array<unsigned long> csrcs;
>>>
>>>
>>>      bytes payload;
>>>
>>>     }
>>>
>>>
>>>     dictionary RtpHeaderExtension {
>>>
>>>      unsigned int id;
>>>
>>>      bytes value;
>>>
>>>     }
>>>
>>>
>>>     dictionary RtcpPacket {
>>>
>>>      // ...
>>>
>>>     }
>>>
>>>
>>>     A streams interface:
>>>
>>>
>>>     Interface RtpTransport {
>>>
>>>       ReadableStream<RtpPacket, RtcpPacket> reader;
>>>
>>>       WritableStream<RtpPacket, RtcpPacket> writer;
>>>
>>>     }
>>>
>>>
>>>     (note: Syntax for defining acceptable types for a stream invented
>>>     by hta)
>>>
>>>      
>>>
>>>     *
>>>     -- 
>>>     Surveillance is pervasive. Go Dark.
>>
>>
>

Received on Wednesday, 30 May 2018 17:13:56 UTC