Re: Raw data API - 3 - Encoded data

Having access to WebRTC API in workers can be useful , especially when
video not involved. I've seen apps where UI affects WebRTC performance.

On Thu, May 31, 2018, 1:06 PM Harald Alvestrand <harald@alvestrand.no>
wrote:

> Den 30. mai 2018 21:49, skrev Randell Jesup:
> > On 5/30/2018 12:29 PM, Peter Thatcher wrote:
> >> This gets to my question I want to cover at the f2f (and which we now
> >> have agenda time for): how low-level an API do we want?  With a
> >> low-level RTP transport object, a JS/wasm app could implement a
> >> high-level RtpSender on top.  But is that the right balance between
> >> flexibility and ease of use?  How much do we want to rely on libraries
> >> to fill in the higher-level stuff?
> >>
> >> I feel like providing low-level components and letting the js/wasm
> >> libraries fill in the higher-level stuff is the right way to go for
> >> most things, but finding the right line is a bunch of tradeoffs and I
> >> think there's a lot of the WG to consider.
> >
> > Another thing to consider (and this applies to a lot of the discussions
> > going on for the f2f): JS isn't realtime.  Not even close.  When you
> > look at profiles like I do, and see JS pauses for GC/CC,
> > layout/rendering, other things jumping in, etc...  When one page leaks
> > (I'm looking at you adsafeprotected.com (anti-clickfraud provider) -
> > though I poked them hard and they fixed it finally), you can see LONG
> > GC/CC times.  Like 100's of ms.  or seconds.
> >
> > Right now WebRTC is so isolated from JS land (at least in Firefox) that
> > you can have a call continue for 10 minutes after the mainthread has
> > deadlocked (until consent-refresh kills you, for example).  Push tons of
> > stuff in to JS, and you're going to majorly destabilize timing and
> > latency and framerate.  If you push most of this off into Workers
> > running WASM, you may avoid the random latencies, but at some real costs
> > in making everything multithreaded to the extreme.  And even if you can
> > avoid adding delay or jitter to the media, you're going to be using more
> > RAM and CPU especially on mobile devices.  WASM is good, true, but it's
> > not a panacea.
>
> WASM makes you able to do things faster, but not more consistently.
>
> So yes, making something that works close to realtime requires getting
> off the main thread.
>
> Which means workers. Of some kind.
>
>
>