W3C home > Mailing lists > Public > public-webrtc@w3.org > July 2015

Re: Add "MediaStream with worker" for video processing into the new working items of WebRTC WG

From: Robert O'Callahan <robert@ocallahan.org>
Date: Tue, 28 Jul 2015 23:05:40 +1200
Message-ID: <CAOp6jLbmJzQRXqJyYBuwoi00Yr0auZWnceTFXeMFd2PRY9zDrw@mail.gmail.com>
To: Mathieu Hofman <Mathieu.Hofman@citrix.com>
Cc: "public-media-capture@w3.org" <public-media-capture@w3.org>, "public-webrtc@w3.org" <public-webrtc@w3.org>, Chia-Hung Tai <ctai@mozilla.com>
[Clearing most of the CC list]

I feel very strongly that we should not support transferring MediaStreams
or MediaStreamTracks, especially across Workers. That would add great
complexity to the Gecko MediaStream and WebAudio implementations. Martin
already mentioned the multiprocess issue. Another issue is that proper
memory management for WebAudio nodes and MediaStream(Tracks), e.g. to
garbage-collect nodes, media elements or streams that are no longer
relevant, is already quite difficult, and extending that across thread
boundaries would be a nightmare. I much prefer alternatives like
Chia-Hung's current proposal, which don't require that.

Note: as Martin also mentioned, supporting WebAudio or even MediaStreams in
a worker is not itself much of a problem, if we want to do that. The key is
that all connected nodes and streams should be associated with the same
Worker or window.

As for backpressure, as I understand it, Chia-Hung's proposal does support
a form of backpressure. If a video processing or monitoring callback is
still processing frame N when frame N+1 becomes available, frame N+1 is
queued until the callback completes, but the queue only has length 1, so if
frame N+2 arrives while N is still being processed, frame N+1 is
effectively dropped.

Rob
-- 
lbir ye,ea yer.tnietoehr  rdn rdsme,anea lurpr  edna e hnysnenh hhe uresyf
toD
selthor  stor  edna  siewaoeodm  or v sstvr  esBa  kbvted,t
rdsme,aoreseoouoto
o l euetiuruewFa  kbn e hnystoivateweh uresyf tulsa rehr  rdm  or rnea
lurpr
.a war hsrer holsa rodvted,t  nenh hneireseoouot.tniesiewaoeivatewt sstvr
esn
Received on Tuesday, 28 July 2015 11:06:08 UTC

This archive was generated by hypermail 2.3.1 : Monday, 23 October 2017 15:19:45 UTC