Re: WebRTC in workers

Den 22. juni 2018 05:00, skrev youenn fablet:
> Hi all,
> 
> Following on last F2F meeting, please find below some thoughts on WebRTC in workers.
> This should complete my related action item.
> 
> Exposing WebRTC constructs to workers (including service workers) would benefit some scenarios. The scenarios envisioned today mostly make use of data channel as workers currently lack APIs to handle media content. Some identified benefits are:
> - Having a single and more persistent WebRTC connection if done in service worker
> - Having a more flexible connection than WebSocket
> - Processing data out of the main thread
> 
> WebRTC in workers can be split in different items:
> 1. Enable data channel support in workers
> 2. Enable WebRTC media exchange support in workers
> 3. Enable transferring WebRTC/media objects between different contexts
> 4. Enable processing media content in workers
> 
> Item 1 can be implemented on top of WebRTC 1.0 by exposing the following interfaces to workers, each interface being added a constructor as defined more or less like in ORTC:
> - RTCIceTransport
> - RTCDtlsTransport
> - RTCSctpTransport
> - RTCDataChannel
> This might allow shimming PeerConnection for data channel use.
> 
> Item 2 would require exposing these additional interfaces:
> - RTCRtpSender/RTCRtpReceiver + ORTC-like constructors
> - MediaStreamTrack/MediaStream
> Combined with item1, it might be envisioned to expose RTCPeerConnection natively or as a shim.
> 
> Item 3 covers the possibility to transfer MediaStream/MediaStreamTrack between different contexts (workers, cross origin documents) living in potentially different threads/processes.
> It could also cover the possibility to transfer RTC objects between different contexts.
> 
> Item 4 exposes functionality to process/generate MediaStreams.
> WebAudio could be used to generate/process audio tracks.
> OffscreenCanvas could be used as a basis to generate/process video tracks.
> 
> Item 1 seems valuable as an incremental improvement over WebRTC 1.0.
> Items 2, 3 and 4 need more study in terms of use cases, feasibility…
> 
> Thanks,
>  Y
> 

My immediate thought is that item 4 can be done with only item 3: If I
can transfer a MediaStreamTrack into a worker, and have the APIs for the
MediaStreamTrack exposed there, I can do processing of media content in
workers (MediaStreamTrack in / MediaStreamTrack out).

Worklets are another model for media processing.

>From the description of what you note as requirements for item 1, it
sounds as if you are imagining that the worker would be able to set up
its own data channels with all the transport stuff; another option is to
make RTCDataChannel transferable, and let PeerConnection remain in [window].

So I'd like to split the scenario classification along the axis of
"window sets up connection" and "worker sets up connection" - and see
where our scenarios end up in such a matrix.

Received on Friday, 22 June 2018 09:25:38 UTC