Re: senders vs receivers codec selection

Few considerations regarding codec switching that might affect
implementation:

1. When transmitting we should be able to specify multiple codecs for a
single RTP sender to support things like CN and RFC 4733 tones. Transmitter
should switch to those automatically based on transmitted audio.

2. When playing back audio transmitted using different payloads a smoothed
time-synchronized playback needs to be ensured. In other words, when
switching from CN to OPUS and back same jitter buffer needs to be used and
audio should be continuously played back.

3. Receiver should be able to differentiate between payload switch and
packet loss. In other words, if an audio stream switches from OPUS to CN
and back, receiver should start playback of CN and not used packet loss
concealment for OPUS for the time period when CN is available. Audio codec
state might also need to be reinitialized after each payload switch.

4. We should be able to deal with redundant payload, like RFC 4733 and OPUS
audio, which can be available for some period of time. Only one of the
audio representations should be played back.

I believe that 3 and 4 would require a way for RTP receiver to specify that
several payloads are related and only one of them should be played back at
any given time.

_____________
Roman Shpount


On Mon, May 12, 2014 at 10:21 AM, Robin Raymond <robin@hookflash.com> wrote:

>
> Yes, you are absolutely correct that we must be capable of switching
> between codecs without signaling. It's critical and there are many use case
> even beyond legacy boxes that do that. This is why I suggesting that it
> should work for any codec, not just non-SVC codecs.
>
> But to be fair, there are two ways to do that switch without signaling
> with the model we have:
> a) Each "receiver" object could latches onto 1 codec per SSRC. To receive
> with a different codec on the same SSRC, use a different "receiver" object
> (which will output it's own "media track")
> -or-
> b) Each "receiver" object contains a list of all codecs for a given SSRC
> and the receiver object can auto-switch between them should the sender
> change codecs
>
> Either will work. Option "a" requires the application developer manually
> switch between "media tracks" as each track become active. Option "b" has
> the advantage of re-using the same output "media track" despite the codec
> changing so it doesn't require the application developer to re-wire their
> rendered output every time a codec is changed (i.e. auto-switching). We
> justified including auto-switching for simulcasting within the "receiver"
> object so the receiver can output to the same "media track" without setting
> up new receiver objects per simulcast stream.
>
> This leads to a few questions:
> 1) With our current model, despite having a list of codecs, do we only
> latch to one codec and if the sender changes codecs would that require a
> new "receiver" object if the codec changed?
> 2) If the current model does allow switching between codecs on the same
> SSRC without a new receiver object, can we make it work for SVC codecs too
> and not just non-SVC codecs?
>
> -Robin
>
>   Roman Shpount <rshpount@turbobridge.com>
>  May 12, 2014 at 12:11 AM
> One thing I like to mention is that we do need an ability to receive
> multiple codecs and switch between them based on the payload type to be
> interoperable with current VoIP. To interop with current WebRTC we need to
> support ability to switch between OPUS, RFC 4733 tones and comfort noise
> based on the payload type. I am also familiar with quite a few system which
> switch the transmitting audio codec without any signaling notification to
> the remote side, such as international VoIP carriers that switch between
> G.729 and G.711 within the same call with no signaling.
>
> _____________
> Roman Shpount
>
>
>
>   Robin Raymond <robin@hookflash.com>
>  May 11, 2014 at 8:24 PM
>
> Currently the receiver has a payload type for each codec. This
> configuration allows the receiver can receiver with any codec on the same
> SSRC and allow auto-switching between them. This ability is true so long as
> the codec is not SVC enabled because the layering information was defined
> within the context of the receiver object itself and not within the context
> of receiver's codec.
>
> If the receiver object is only supposed to work with a particular codec
> only then why are we supplying the receiver object with a set of codecs and
> payload type for each codec? So to me, that suggests the receiver object
> CAN switch between codecs. If does allow this behaviour then why doesn't it
> then work too for SVC codecs but only non SVC codecs?
>
> To me this implies we should do one of the following:
> 1) make the sender / receiver only have one codec and that's it. Want to
> receive or send with a different codec? Use a different sender / receiver
> object. -OR-
> 2) make the sender / receiver work for all codecs equally by defining the
> layering within the context of the SVC codec and not within the context of
> the receiver
>
> I'm completely fine with either scenario "1" or "2" but I don't like the
> way it kind-of half works now. "2" is more work for ORTC implementors but
> it allows for easier management of the receiving streams for an application
> developer.
>
> -Robin
>
>