[webrtc-pc] Opus mono/stereo and remote track channelCount (#3010)

henbos has just created a new issue for https://github.com/w3c/webrtc-pc:

== Opus mono/stereo and remote track channelCount ==
Opus is a codec that is capable of both mono and stereo.

In SDP, you can specify "stereo=X" to say what your receiver preference is, but importantly:
- This is just a preference, regardless what the value of X is and regardless if "stereo=" is even present in the SDP, the decoder has to be prepared for both mono and stereo input on the wire. It's completely optional in every way.

Today in Chrome, you get mono (track.getSettings().channelCount == 1) by default, but if "stereo=1" is used you get stereo (channelCount == 2). Importantly, the decoder will downmix or upmix to match the desired output (e.g. if stereo is used you will get a stereo track even if the input on the wire is mono).

We would like to support stereo without SDP munging. The question is what is the expected behavior?
1. Opus is a stereo codec, so always output stereo regardless of signal on the wire.
2. Adjust the output of the decoder to match what is on the wire (mono = mono, stereo = stereo). This could change on the fly.

While I originally thought 2) was more intuitive, I'm told that Firefox does 1). It also turns out that the audio team prefers to do 1) to avoid some complexity and to reduce the risk of audio glitches due to reconfiguring the decoder in the event that the signal switches back and forth on the fly (use case: virtual audio ssrc and speaker switching between mono and stereo signals).

**Proposal A**: Mandate "always stereo" for stereo codecs.
**Proposal B**: Mandate dynamically switching between mono and stereo.
**Proposal C**: Mandate nothing; it is up to the user agent.

Please view or discuss this issue at https://github.com/w3c/webrtc-pc/issues/3010 using your GitHub account


-- 
Sent via github-notify-ml as configured in https://github.com/w3c/github-notify-ml-config

Received on Tuesday, 22 October 2024 11:49:38 UTC