Re: Raw data API - 4 - direct RTP access

Den 29. mai 2018 13:04, skrev Sergio Garcia Murillo:
> My main concern with this API is that we don't have a clear scope on
> what we intend to do on javascript and what should be provided by the
> browser stack.

My previous posting on use cases didn't elicit all that much debate -
the relevant section for this particular API is this one:

"The same APIs are needed for other functions, such as:

    ML-NetEq: Jitter buffer control in other ways than the built-in browser

        This also needs the ability to turn off the built-in jitter
buffer, and therefore makes this API have the same timing requirements
as dealing with raw data

    ML-FEC: Application-defined strategies for recovering from lost packets.

    Alternative transmission: Using something other than browser’s
built-in realtime transport (currently SRTP) to move the media data
"


> 
> As Bernard has already mentioned, is FEC/RTX meant to be implemented on
> the js layer on the stack or a mix of both? If you decide to implement
> the FEC on the stack, how do you allow to receive RTCP nacks on the js
> layer or being handled by the stack? Do I want to manually send the RTCP
> NACKs for missing packets or not? The  Same with bandwidth management,
> how do we control/prevent what is the flow of the rtcp messages desired?

An application that does FEC or NetEq in application-space will
naturally have to have the required signals available.
What the application takes responsibility for, the application will have
to take responsibility for.

With bandwidth management, the browser needs to police the overall usage
to protect the net (and other users) - but if the application wants to
redistribute the packets sent between media that is under a single
congestion controller - it should not be a concern of the browser.

> The amount of switches and configurations parameters to control all of
> that is just too huge to be usable.

So far, I haven't suggested a single switch or configuration parameter
in the APIs proposed, so whether there will be too many is entirely unclear.

When I look at the present PeerConnection API in the abstract, I
certainly think that we have too many switches and configuration
parameters in one place. But we had WG consensus on adding every single
one of them. Maybe they're just in the wrong place.

> IMHO we should target to a "media
> only RTP" api (with no rtcp) or a full RTP/RTCP apis delegating all the
> responsibility to the js layer.

This (thread 4) is the full RTP/RTCP API.

The previous posting (thread 3) is a "media only" API.

We can discuss them separately.

> 
> Best regards
> Sergio
> 
> On 29/05/2018 8:31, Harald Alvestrand wrote:
>>
>> **
>>
>>
>>   *Low-level RtpTransport*
>>
>> *
>>
>> One alternative to extending the RTPSender interface is to define a
>> new RTPTransport interface that takes RTP packets as its input/output
>> format. This would allow applications that desire to do so to
>> implement the entire RTP stack (or a tweaked RTP stack) in Javascript
>> or WebAssembly.
>>
>>
>> Note: before deciding this level, we have to decide whether congestion
>> control and encryption lives above or below the interface. If we do an
>> API that does encryption in user space, we also expose keys to
>> Javascript, which is not always a Good Thing.
>>
>> The transport has to police the congestion control, so the interface
>> has to take congestion control signals from the transport and take
>> appropriate action in choosing to send or not to send.
>>
>> This (API for congestion control) is said to be an advantage of the
>> Streams API, so this should at least be looked at.
>>
>>
>> An example interface:
>>
>>
>> // A DtlsTransport and/or IceTransport underneath
>>
>> // This is encrypted and congestion-controlled
>>
>> interface RtpTransport {
>>
>>  sendRtpPacket(RtpPacket packet);
>>
>>  sendRtcpPacket(RtcpPacket packet);
>>
>>  attribute eventhandler onrtppacket;
>>
>>  attribute eventhandler onrtcppacket;
>>
>> }
>>
>>
>> dictionary RtpPacket {
>>
>>  // Metadata/"control block"
>>
>>  unsigned octet payloadType;
>>
>>  unsigned int sequenceNumber;
>>
>>  unsigned long ssrc;
>>
>>  array<RtpHeaderExtension> header_extensions;
>>
>>  array<unsigned long> csrcs;
>>
>>
>>  bytes payload;
>>
>> }
>>
>>
>> dictionary RtpHeaderExtension {
>>
>>  unsigned int id;
>>
>>  bytes value;
>>
>> }
>>
>>
>> dictionary RtcpPacket {
>>
>>  // ...
>>
>> }
>>
>>
>> A streams interface:
>>
>>
>> Interface RtpTransport {
>>
>>   ReadableStream<RtpPacket, RtcpPacket> reader;
>>
>>   WritableStream<RtpPacket, RtcpPacket> writer;
>>
>> }
>>
>>
>> (note: Syntax for defining acceptable types for a stream invented by hta)
>>
>>  
>>
>> *
>> -- 
>> Surveillance is pervasive. Go Dark.
> 
>

Received on Tuesday, 29 May 2018 11:19:55 UTC