Re: Raw data API - 4 - direct RTP access

On Wed, May 30, 2018 at 9:43 AM Sergio Garcia Murillo <
sergio.garcia.murillo@gmail.com> wrote:

>
> What about RTCP SR/RR/XR?
>

App.  The apps can add things without waiting for all the browsers to add
them.


> I think that should also be handled by the core as well. And NACks, I
> can't imagine why an app would want to deal/mangle with them.
>

Maybe you don't want to send them at all and you want to rely on ACKs
instead (transport-cc already adds ACKs; could just use those).


>
> Also, (Flex) FEC should be quite integrated with congestion control as it
> should be the preferred way of bandwidth probing,
>

Network probing is an interesting question.  But it doesn't have to be
FEC.  It could be aggressive RTX, or something else the app comes up with.
The important part here isn't that the transport knows about FEC or RTX.
It's that the user of the transport can provide some kind of priority flag
("send this if you need to probe but otherwise toss it" type of thing).


> it will be very difficult to make them work together if one is implemented
> in the core and the other in js/wasm.
>

Nah.  You just need what I suggested: a way to indicate packets are for
probing purposes and or low priority.


>
> IMHO, everything that is standard, should be implemented by the core and
> not allow the app to play with it.
>

You're advocating, then, for a high-level API.   On the flip side, an app
should be allowed to do many non-standard or custom things as well.  And
that requires a low-level API.  And that's the root of the tension between
high- and low-level APIs: if you can control more, you have to do more.

What objection do you have in these things being provided by libraries
instead of being baked in the browser?




Best regards
>
> Sergio
>
>
> On 30/05/2018 18:32, Peter Thatcher wrote:
>
> I think there are many levels of RTP transport we can consider, and we
> should consider the lowest reasonable level as well, in which the browser
> would provide:
>
> 1.  Packet serialization (from struct to packet)
> 2.  Congestion control (perhaps by adding its own header extensions or
> RTCP messages)
> 3.  Crypto (SRTP)
>
> And that's it.
>
> I don't think it's worth considering anything without crypto or congestion
> control.
>
> Higher-level transport could also be considered, but I think this
> low-level one would be a good one.
>
> On Tue, May 29, 2018 at 4:08 AM Sergio Garcia Murillo <
> sergio.garcia.murillo@gmail.com> wrote:
>
>> My main concern with this API is that we don't have a clear scope on what
>> we intend to do on javascript and what should be provided by the browser
>> stack.
>>
>> As Bernard has already mentioned, is FEC/RTX meant to be implemented on
>> the js layer on the stack or a mix of both? If you decide to implement the
>> FEC on the stack, how do you allow to receive RTCP nacks on the js layer or
>> being handled by the stack? Do I want to manually send the RTCP NACKs for
>> missing packets or not? The  Same with bandwidth management, how do we
>> control/prevent what is the flow of the rtcp messages desired?
>>
>> The amount of switches and configurations parameters to control all of
>> that is just too huge to be usable. IMHO we should target to a "media only
>> RTP" api (with no rtcp) or a full RTP/RTCP apis delegating all the
>> responsibility to the js layer.
>>
>> Best regards
>>
>> Sergio
>>
>> On 29/05/2018 8:31, Harald Alvestrand wrote:
>>
>> *Low-level RtpTransport*
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> * One alternative to extending the RTPSender interface is to define a new
>> RTPTransport interface that takes RTP packets as its input/output format.
>> This would allow applications that desire to do so to implement the entire
>> RTP stack (or a tweaked RTP stack) in Javascript or WebAssembly. Note:
>> before deciding this level, we have to decide whether congestion control
>> and encryption lives above or below the interface. If we do an API that
>> does encryption in user space, we also expose keys to Javascript, which is
>> not always a Good Thing. The transport has to police the congestion
>> control, so the interface has to take congestion control signals from the
>> transport and take appropriate action in choosing to send or not to send.
>> This (API for congestion control) is said to be an advantage of the Streams
>> API, so this should at least be looked at. An example interface: // A
>> DtlsTransport and/or IceTransport underneath // This is encrypted and
>> congestion-controlled interface RtpTransport {  sendRtpPacket(RtpPacket
>> packet);  sendRtcpPacket(RtcpPacket packet);  attribute eventhandler
>> onrtppacket;  attribute eventhandler onrtcppacket; } dictionary RtpPacket {
>>  // Metadata/"control block"  unsigned octet payloadType;  unsigned int
>> sequenceNumber;  unsigned long ssrc;  array<RtpHeaderExtension>
>> header_extensions;  array<unsigned long> csrcs;  bytes payload; }
>> dictionary RtpHeaderExtension {  unsigned int id;  bytes value; }
>> dictionary RtcpPacket {  // ... } A streams interface: Interface
>> RtpTransport {   ReadableStream<RtpPacket, RtcpPacket> reader;
>>   WritableStream<RtpPacket, RtcpPacket> writer; } (note: Syntax for
>> defining acceptable types for a stream invented by hta)   *
>>
>> --
>> Surveillance is pervasive. Go Dark.
>>
>>
>>
>