Re: Raw data API - 6 - Encoders/decoders

On 14/06/2018 11:55, Peter Thatcher wrote:
>
> "WebRTC developers" are not a homogenous group.  I know of many that 
> have things they are doing with mobile/native/server endpoints that 
> they can't do with web endpoints because the Web API isn't 
> sufficiently capable.
>
> We started ORTC years ago to try and provide something better.  Our 
> experience over the last few years with more mobile and server 
> endpoints has taught us lessons of how ORTC can be improved upon.   So 
> while I think it's a good starting point for WebRTC NV, I think we can 
> do even better.  In particular, two things: 1.  Split RtpSender into 
> encoder and transport (and similar for RtpReceiver) and 2. have more 
> control over ICE.  There are others, but those are the main ones to me.
I fully agree with this.
>
>     So, after failing to provide a high level API that is usable,
>     developers are requesting a lower level API with a sane API, not
>     because they all want to deal with raw rtp packets, do funny hats
>     or send rtcp, but because we want to use something that is
>     simpler, more fine grained and with a better control surface. Some
>     of them may want to also perform raw operations in one way or
>     another, so we should also support that use cases.
>
>
> That's the key: finding the right balance between allowing apps to do 
> low-level control when they need to and not requiring them to do so 
> when they don't want to.  It's a difficult balance to find, which is 
> why so much of my effort, and my presentation time at the f2f, is 
> focused on use cases an finding this balance.

Splitting rtpsender in encoder/decoder and transport objects and provide 
a finer grained apis to control then seems something that we can all 
easily agree on. But devil is in the details, and IMHO we have jumped 
into the wasm stuff and api examples too soon.

Regardless of the API surface we choose in the end, my point is that we 
need to define two modes of operation, a stream-like piped approach in 
which all components are provided by the browsers, and raw access to 
rtp/media streams to perform the use cases that required it. Just 
providing the later and expecting the web developers to either 
reimplement an rtp stack in js/wasm or use old webrtc apis is a no-go IMHO.

Best regards

Sergio