Re: Raw data API - 4 - direct RTP access

I think my concerns are more about overriding the rtp browser stack and 
re-implementing the same stuff on javascript. For example in a less 
controversial example, should the application be allowed to deal with 
the RTCP raw sender/receiver reports?

Regarding my "media only rtp", I was not referring to the media frame 
api, but more like a "userland-rtp" mode, in which the rtp browser stack 
would still perform automatically some stuff (like rtx, fec, nacks, 
sender/receiver reports, plis, etc). You could turn off that features in 
the browser stack (for example turn down the rtx) and implement the same 
in "userland". We could even allow the js to send/receive rtcp data in a 
specific rtcp application layer feedback specific type.

The API would be very similar, and the uses cases covered also, but the 
complexity would be reduced drastically imho.

Best regards
Sergio

On 29/05/2018 13:19, Harald Alvestrand wrote:
> Den 29. mai 2018 13:04, skrev Sergio Garcia Murillo:
>> My main concern with this API is that we don't have a clear scope on
>> what we intend to do on javascript and what should be provided by the
>> browser stack.
> My previous posting on use cases didn't elicit all that much debate -
> the relevant section for this particular API is this one:
>
> "The same APIs are needed for other functions, such as:
>
>      ML-NetEq: Jitter buffer control in other ways than the built-in browser
>
>          This also needs the ability to turn off the built-in jitter
> buffer, and therefore makes this API have the same timing requirements
> as dealing with raw data
>
>      ML-FEC: Application-defined strategies for recovering from lost packets.
>
>      Alternative transmission: Using something other than browser’s
> built-in realtime transport (currently SRTP) to move the media data
> "
>
>
>> As Bernard has already mentioned, is FEC/RTX meant to be implemented on
>> the js layer on the stack or a mix of both? If you decide to implement
>> the FEC on the stack, how do you allow to receive RTCP nacks on the js
>> layer or being handled by the stack? Do I want to manually send the RTCP
>> NACKs for missing packets or not? The  Same with bandwidth management,
>> how do we control/prevent what is the flow of the rtcp messages desired?
> An application that does FEC or NetEq in application-space will
> naturally have to have the required signals available.
> What the application takes responsibility for, the application will have
> to take responsibility for.
>
> With bandwidth management, the browser needs to police the overall usage
> to protect the net (and other users) - but if the application wants to
> redistribute the packets sent between media that is under a single
> congestion controller - it should not be a concern of the browser.
>
>> The amount of switches and configurations parameters to control all of
>> that is just too huge to be usable.
> So far, I haven't suggested a single switch or configuration parameter
> in the APIs proposed, so whether there will be too many is entirely unclear.
>
> When I look at the present PeerConnection API in the abstract, I
> certainly think that we have too many switches and configuration
> parameters in one place. But we had WG consensus on adding every single
> one of them. Maybe they're just in the wrong place.
>
>> IMHO we should target to a "media
>> only RTP" api (with no rtcp) or a full RTP/RTCP apis delegating all the
>> responsibility to the js layer.
> This (thread 4) is the full RTP/RTCP API.
>
> The previous posting (thread 3) is a "media only" API.
>
> We can discuss them separately.
>
>> Best regards
>> Sergio
>>
>> On 29/05/2018 8:31, Harald Alvestrand wrote:
>>> **
>>>
>>>
>>>    *Low-level RtpTransport*
>>>
>>> *
>>>
>>> One alternative to extending the RTPSender interface is to define a
>>> new RTPTransport interface that takes RTP packets as its input/output
>>> format. This would allow applications that desire to do so to
>>> implement the entire RTP stack (or a tweaked RTP stack) in Javascript
>>> or WebAssembly.
>>>
>>>
>>> Note: before deciding this level, we have to decide whether congestion
>>> control and encryption lives above or below the interface. If we do an
>>> API that does encryption in user space, we also expose keys to
>>> Javascript, which is not always a Good Thing.
>>>
>>> The transport has to police the congestion control, so the interface
>>> has to take congestion control signals from the transport and take
>>> appropriate action in choosing to send or not to send.
>>>
>>> This (API for congestion control) is said to be an advantage of the
>>> Streams API, so this should at least be looked at.
>>>
>>>
>>> An example interface:
>>>
>>>
>>> // A DtlsTransport and/or IceTransport underneath
>>>
>>> // This is encrypted and congestion-controlled
>>>
>>> interface RtpTransport {
>>>
>>>   sendRtpPacket(RtpPacket packet);
>>>
>>>   sendRtcpPacket(RtcpPacket packet);
>>>
>>>   attribute eventhandler onrtppacket;
>>>
>>>   attribute eventhandler onrtcppacket;
>>>
>>> }
>>>
>>>
>>> dictionary RtpPacket {
>>>
>>>   // Metadata/"control block"
>>>
>>>   unsigned octet payloadType;
>>>
>>>   unsigned int sequenceNumber;
>>>
>>>   unsigned long ssrc;
>>>
>>>   array<RtpHeaderExtension> header_extensions;
>>>
>>>   array<unsigned long> csrcs;
>>>
>>>
>>>   bytes payload;
>>>
>>> }
>>>
>>>
>>> dictionary RtpHeaderExtension {
>>>
>>>   unsigned int id;
>>>
>>>   bytes value;
>>>
>>> }
>>>
>>>
>>> dictionary RtcpPacket {
>>>
>>>   // ...
>>>
>>> }
>>>
>>>
>>> A streams interface:
>>>
>>>
>>> Interface RtpTransport {
>>>
>>>    ReadableStream<RtpPacket, RtcpPacket> reader;
>>>
>>>    WritableStream<RtpPacket, RtcpPacket> writer;
>>>
>>> }
>>>
>>>
>>> (note: Syntax for defining acceptable types for a stream invented by hta)
>>>
>>>   
>>>
>>> *
>>> -- 
>>> Surveillance is pervasive. Go Dark.
>>
>

Received on Tuesday, 29 May 2018 12:10:10 UTC