[webrtc-pc] Are sinks a prerequisite for decoding? getSynchronizationSources() without <video> (#2240)

henbos has just created a new issue for https://github.com/w3c/webrtc-pc:

== Are sinks a prerequisite for decoding? getSynchronizationSources() without <video> ==
When [Chrome tried to update its getSynchronizationSources() implementation](https://bugs.chromium.org/p/webrtc/issues/detail?id=10545#c14) to match [the spec](https://w3c.github.io/webrtc-pc/#dom-rtcrtpreceiver-getsynchronizationsources) about when the information is updated...

> When an audio or video frame from one or more RTP packets is delivered to the RTCRtpReceiver's MediaStreamTrack, the user agent MUST queue a task to update the relevant information for the RTCRtpContributingSource and RTCRtpSynchronizationSource dictionaries based on the content of those packets. 

... it started failing the [getSynchronizationSources() tests](https://cs.chromium.org/chromium/src/third_party/blink/web_tests/external/wpt/webrtc/RTCRtpReceiver-getSynchronizationSources.https.html) because without the MediaStreamTrack having a sink, Chrome was performing optimizations not to decode frames, and so it wasn't updating the SSRC information.

The question is: Is this spec compliant?

> As stated in the conformance section, requirements phrased as algorithms may be implemented in any manner so long as the end result is equivalent. So, an implementation does not need to literally queue a task for every frame, as long as the end result is that within a single event loop task execution, all returned RTCRtpSynchronizationSource and RTCRtpContributingSource dictionaries for a particular RTCRtpReceiver contain information from a single point in the RTP stream.

I would expect the MediaStreamTrack to, at least conceptually, receive frames as long as we are receiving decodable packets, whether or not the track forwards those frames to a sink such as a &lt;video&gt; tag.

On the other hand, why decode packets if we're not using them?

So...
1. Should we add a note to the spec saying "if the MediaStreamTrack does not have any sinks, this step MAY be optimized away, so that the SSRC information is not updated"?
2. Or should getSSRC be clarified to make the packet information mandatory even if there are no sinks and no decoding happening?
3. Or should decoding be mandatory, leaving no ambiguity here?

Please view or discuss this issue at https://github.com/w3c/webrtc-pc/issues/2240 using your GitHub account

Received on Friday, 26 July 2019 09:45:01 UTC