Re: webrtc nv and whatwg streams

On 06/10/2018 02:13 AM, Randell Jesup wrote:
> On 6/9/2018 2:52 PM, Sergio Garcia Murillo wrote:
>> On 09/06/2018 19:41, Randell Jesup wrote:
>>> Not to belabor a point here, but if this is to occur on the main
>>> thread there's a real problem with this API (and this could occur I
>>> suppose, with a bunch of additional work elsewhere in the spec, on a
>>> Worker). 
>>>
>>> Even on a Worker, outside of emscriptem/WASM code that carefully
>>> doesn't throw garbage, there will be GC pauses causing hiccups in
>>> the stream - potentially long (in a realtime sense).
>>>
>>
>> Fully agree. But if all components are implemented natively there
>> should be no need to go into the main thread or call any js code once
>> the streams are attached and the pipe chain is created. If someone
>> wants to reimplement any of the components in js or add custom ones,
>> then, they will have to be really careful, but they will not impact
>> on the performance of the rest of the people using the APIs.
>>
>> Not sure what is the expected whatwg stream behavior and if this is
>> supported/possible, but both in their use cases and the design would
>> be quite similar to what we are trying to achieve.
>
> Correct - if these are all predefined nodes (or something in WASM that
> doesn't throw garbage, probably), then you won't be subject to GC (or
> event queue) delays.  I may have been jumping the gun on suggesting
> that this was a problem with streams here (it's a saturday and I'm
> packing for a conference).  If, however, any of these takes a trip
> through JS/event queues, then it's problematic (like a number of the
> proposals that had JS involved in packetization and other network and
> media-input related actions).

It is a problem with letting users define arbitrary streams, because
users defining arbitrary streams can cause JS events at arbitrary points.
It is not a problem with exposing "something that you can think of as a
stream" (if we couple pre-made streams together and short-circuit the
stuff that underlies them) - but in that case, we're only playing around
with the shape of the control surface for the stream, we're not doing
anything to provide new functionality.

The streams model has a core problem with copies: A source stream vends
its own objects, which requires buffer copies when processing the data
elements in WASM. This is mitigated for byte streams using the BYOB
(bring your own buffer) model, but no mitigation is available for
non-byte streams like streams of video frames.


> -- 
> Randell Jesup -- rjesup a t mozilla d o t com
> Please please please don't email randell-ietf@jesup.org!  Way too much spam


-- 
Surveillance is pervasive. Go Dark.