W3C home > Mailing lists > Public > public-webrtc-logs@w3.org > July 2019

Re: [webrtc-pc] Are sinks a prerequisite for decoding? getSynchronizationSources() without <video> (#2240)

From: Chen Xing via GitHub <sysbot+gh@w3.org>
Date: Mon, 29 Jul 2019 11:30:13 +0000
To: public-webrtc-logs@w3.org
Message-ID: <issue_comment.created-515954164-1564399812-sysbot+gh@w3.org>
This seem to be a side-effect of having a "frame pulling" architecture vs. a "frame pushing" one and goes beyond the implementation details of `getSynchronizationSources()` itself.

Video uses the "frame pushing" approach in Chromium. RTP packets arrive. They get decoded. Frames are then "pushed" to the track along with a timestamp of when it should be rendered. This is why the video web tests continued to pass after moving the gathering of info for `GetSynchronizationSources()` from packet receive time to frame delivery time:

https://bugs.chromium.org/p/webrtc/issues/detail?id=10545#c8

Audio uses the "frame pulling" approach in Chromium. RTP packets arrive. They get queued up in the jitter buffer in NetEq. They remain in the jitter buffer until the sound card "pulls" a 10 ms audio frame by ultimately calling `NetEq::GetAudio()`. Without a proper sink, the call is never made and packets stay in the buffer indefinitely. Or more commonly, the packets get discarded when the jitter buffer reaches capacity and is flushed by subsequent `NetEq::InsertPacket()` calls. This why the audio web tests started to fail after attempting to move the gathering of info for `GetSynchronizationSources()` from packet receive time to frame delivery time:

https://bugs.chromium.org/p/webrtc/issues/detail?id=10545#c12

This behavior is also noticeable elsewhere. We can take a look at the WebRTC sample named "Peer connection as input to Web Audio":

https://webrtc.github.io/samples/src/content/peerconnection/webaudio-output/

Pressing "Start" followed by "Call" shows a visualization of the audio from the microphone. But if we go to "main.js" and remove the line that says `remoteVideo.srcObject = e.streams[0];` from `function gotRemoteStream(e)`, we suddenly get nothing with Chromium. Firefox works either way.

-- 
GitHub Notification of comment by chxg
Please view or discuss this issue at https://github.com/w3c/webrtc-pc/issues/2240#issuecomment-515954164 using your GitHub account
Received on Monday, 29 July 2019 11:30:14 UTC

This archive was generated by hypermail 2.4.0 : Friday, 17 January 2020 19:22:26 UTC