Re: Raw data API - 6 - Encoders/decoders

On Thu, 14 Jun 2018 10:18:34 +0200
Sergio Garcia Murillo <sergio.garcia.murillo@gmail.com> wrote:

> On 14/06/2018 4:35, Peter Thatcher wrote:
> > Yes, that's a valid concern.  But that concern is the same as for
> > any time the app gets in the media flow, such as when it does
> > processing for funny hats or adds encryption.  It might take some
> > implementation experience to figure out where the really perf
> > issues are. [...]
> >  If the app can't process things per-frame, then you might as well 
> > just go back to having an RtpSender like in ORTC that doesn't
> > require the app to do anything on the media path.
> >  [...]
> > But that requires every piece to be implemented by the browser, and
> > I don't see how that buys us much vs. an ORTC-style RtpSender.   It 
> > would be much better if we could find a way to make wasm/js
> > performant in the media path.
> > [...]
> > It would be easy to make the encoders and decoders use WHATWG
> > streams instead of events.  I just don't see the benefit of having
> > an encoder stream tied to a transport stream with no app in between
> > except plugging it together and then hoping that it will be
> > performant because we expect a sufficiently smart browser
> > implementation.  
> 
> I think we have a mayor disagreement here. Your assumption is that 
> webrtc wants the low level APIs because they want to do funny things 
> with the raw data. IMHO that is not true.
> 
> WebRTC developers just hate SDP, dislike transceivers, senders and 
> receivers and trying to get current APIs to work is a pain even for
> the simple cases. ORTC is just a bit better, but not much, with the
> encoding parameters stuff being just a  json-version of an m-line,
> not much improvement there.
> 
> So, after failing to provide a high level API that is usable,
> developers are requesting a lower level API with a sane API, not
> because they all want to deal with raw rtp packets, do funny hats or
> send rtcp, but because we want to use something that is simpler, more
> fine grained and with a better control surface. Some of them may want
> to also perform raw operations in one way or another, so we should
> also support that use cases.
> 


+1 on all of the above...


> In that regards the using whatwg streams provides a simple API
> surface with finer grained control of each of the components in the
> pipe line. The same stream API will most probably used in other areas
> of the browser as well, so the web developers will already have
> knowledge about it. Also, it allows to implement  the raw use cases
> that we were trying to implement, even some of them (custom encoders,
> funny hats) are explicitly mentioned in their documentation as a
> target of the API.
> 
> Does this API provide any advantage over the ORTC-sytle RTPSender if
> we just don't want to do any raw processing? Yes, I would even
> re-implement a standard rtp stack on js than having to use the ORTC
> sender/receiver stuff.
> 


-1 on an RTP stack in JS :-P

I'm strongly against all this wasm nonsense. Just because machines are
getting more powerful, doesn't mean we should drop more efficient code
for something that runs worse just because it's easier to move around.
It's a trend that is happening everywhere, and I hate it, because it
shows no regard for those who still ship machines that would atthe very
least struggle with that. Besides, this seems to bring us back to the
"WebRTC just for browsers" era: WebRTC is everywhere, now, and I can
already picture my mobile phone melting down any time I open a web page
with WebRTC on it.

Even if all that weren't an issue, I still have strong concerns about
the impact this would have on interoperability. Right now, no matter
what you're using, as long as you implement the right protocols things
can exchange media with each other. As Sergio said in another response,
the ability to do your own transport protocol over whatever, or even
your own RTP and packetization for that matter, will very likely lead to
isolated silos that will only talk to what you implement. I'm sure
that's just fine for the likes of Google, Cisco, etc.; very less so for
pretty much everyone else. As a WebRTC server developer, I'm most
definitely not going to catch up with any new "better than anything
else around" protocol that would come out of that.

If "funny hats" and the like are the target here, as explained by
others I think there are better (and basically ready) ways to accomplish
this without disrupting the work that has been done in the last years.


> Last point, the whatwg group has been working on the API for years
> now, so we can leverage on their work and cut down the time to market
> of the webrtc nv API.
> 
> Let's put it the other way, what are the benefits of your API
> compared to the whatwg streams?
> 
> Best regards
> Sergio
> 
> 

I think Peter answered that with the "requires every piece to be
implemented by the browser" argument, which is not a good enough reason
for me. I understand that maintaining a WebRTC stack in C/C++ is
complicated, but it's worth it. WebRTC hid most of the lowest level
stuff (e.g., sending and receiving UDP) within the browser for many
reasons (like preventing DoS attacks originated by JavaScript), and all
this wasm nonsense to do everything in JavaScript brings us several
steps back, meaning that at the very least those same arguments should
be dealt with once more before anything happens.

Of course, just my two cents,
Lorenzo

-- 
I'm getting older but, unlike whisky, I'm not getting any better
https://twitter.com/elminiero

Received on Thursday, 14 June 2018 08:55:54 UTC