W3C home > Mailing lists > Public > public-webrtc@w3.org > June 2018

Re: webrtc nv and whatwg streams

From: Randell Jesup <randell-ietf@jesup.org>
Date: Sat, 9 Jun 2018 13:41:49 -0400
To: public-webrtc@w3.org
Message-ID: <cef64190-44e1-a128-b59a-278d2d91734f@jesup.org>
On 6/9/2018 4:23 AM, Sergio Garcia Murillo wrote:
> I have been reading the whatwg stream api
> https://streams.spec.whatwg.org and it seems it already covers quite a 
> lot of our use cases. Copy&paste follows:
>
> By providing the foundation for these streams to be exposed to 
> developers, the Streams Standard enables use cases like:
>
>   * Video effects: piping a readable video stream through a transform
>     stream that applies effects in real time.
>
> Some examples of potential transform streams includes
>
>   * A video decoder, to which encoded bytes are written and from which
>     uncompressed video frames are read
>
> wouldn't make sense to try to leverage their work?
>
> source=(stream)=>encoder=(stream)=>packetizer=(stream)=>rtpsender=(stream)=>dtls/ice 
> transport
> or
> source=(stream)=>encoder=(stream)=>quic

Not to belabor a point here, but if this is to occur on the main thread 
there's a real problem with this API (and this could occur I suppose, 
with a bunch of additional work elsewhere in the spec, on a Worker).

Even on a Worker, outside of emscriptem/WASM code that carefully doesn't 
throw garbage, there will be GC pauses causing hiccups in the stream - 
potentially long (in a realtime sense).

This especially applies to anything wanting to do transforms to the 
data.  We did propose some APIs for doing transforms on video/image data 
in efficient ways, though that has stalled.


-- 
Randell Jesup -- rjesup a t mozilla d o t com
Please please please don't email randell-ietf@jesup.org!  Way too much spam
Received on Saturday, 9 June 2018 17:44:53 UTC

This archive was generated by hypermail 2.4.0 : Friday, 17 January 2020 19:18:42 UTC