Re: webrtc nv and whatwg streams

On 09/06/2018 19:41, Randell Jesup wrote:
> On 6/9/2018 4:23 AM, Sergio Garcia Murillo wrote:
>> I have been reading the whatwg stream api
>> https://streams.spec.whatwg.org and it seems it already covers quite 
>> a lot of our use cases. Copy&paste follows:
>>
>> By providing the foundation for these streams to be exposed to 
>> developers, the Streams Standard enables use cases like:
>>
>>   * Video effects: piping a readable video stream through a transform
>>     stream that applies effects in real time.
>>
>> Some examples of potential transform streams includes
>>
>>   * A video decoder, to which encoded bytes are written and from
>>     which uncompressed video frames are read
>>
>> wouldn't make sense to try to leverage their work?
>>
>> source=(stream)=>encoder=(stream)=>packetizer=(stream)=>rtpsender=(stream)=>dtls/ice 
>> transport
>> or
>> source=(stream)=>encoder=(stream)=>quic
>
> Not to belabor a point here, but if this is to occur on the main 
> thread there's a real problem with this API (and this could occur I 
> suppose, with a bunch of additional work elsewhere in the spec, on a 
> Worker).
>
> Even on a Worker, outside of emscriptem/WASM code that carefully 
> doesn't throw garbage, there will be GC pauses causing hiccups in the 
> stream - potentially long (in a realtime sense).
>

Fully agree. But if all components are implemented natively there should 
be no need to go into the main thread or call any js code once the 
streams are attached and the pipe chain is created. If someone wants to 
reimplement any of the components in js or add custom ones, then, they 
will have to be really careful, but they will not impact on the 
performance of the rest of the people using the APIs.

Not sure what is the expected whatwg stream behavior and if this is 
supported/possible, but both in their use cases and the design would be 
quite similar to what we are trying to achieve.

Best regards
Sergio