Re: senders vs receivers codec selection

Agree with Roman's points. The receiver should be able to switch codecs
without reinitialization.


On Mon, May 12, 2014 at 11:47 AM, Roman Shpount <rshpount@turbobridge.com>wrote:

> Few considerations regarding codec switching that might affect
> implementation:
>
> 1. When transmitting we should be able to specify multiple codecs for a
> single RTP sender to support things like CN and RFC 4733 tones. Transmitter
> should switch to those automatically based on transmitted audio.
>
> 2. When playing back audio transmitted using different payloads a smoothed
> time-synchronized playback needs to be ensured. In other words, when
> switching from CN to OPUS and back same jitter buffer needs to be used and
> audio should be continuously played back.
>
> 3. Receiver should be able to differentiate between payload switch and
> packet loss. In other words, if an audio stream switches from OPUS to CN
> and back, receiver should start playback of CN and not used packet loss
> concealment for OPUS for the time period when CN is available. Audio codec
> state might also need to be reinitialized after each payload switch.
>
> 4. We should be able to deal with redundant payload, like RFC 4733 and
> OPUS audio, which can be available for some period of time. Only one of the
> audio representations should be played back.
>
> I believe that 3 and 4 would require a way for RTP receiver to specify
> that several payloads are related and only one of them should be played
> back at any given time.
>
> _____________
> Roman Shpount
>
>
> On Mon, May 12, 2014 at 10:21 AM, Robin Raymond <robin@hookflash.com>wrote:
>
>>
>> Yes, you are absolutely correct that we must be capable of switching
>> between codecs without signaling. It's critical and there are many use case
>> even beyond legacy boxes that do that. This is why I suggesting that it
>> should work for any codec, not just non-SVC codecs.
>>
>> But to be fair, there are two ways to do that switch without signaling
>> with the model we have:
>> a) Each "receiver" object could latches onto 1 codec per SSRC. To receive
>> with a different codec on the same SSRC, use a different "receiver" object
>> (which will output it's own "media track")
>> -or-
>> b) Each "receiver" object contains a list of all codecs for a given SSRC
>> and the receiver object can auto-switch between them should the sender
>> change codecs
>>
>> Either will work. Option "a" requires the application developer manually
>> switch between "media tracks" as each track become active. Option "b" has
>> the advantage of re-using the same output "media track" despite the codec
>> changing so it doesn't require the application developer to re-wire their
>> rendered output every time a codec is changed (i.e. auto-switching). We
>> justified including auto-switching for simulcasting within the "receiver"
>> object so the receiver can output to the same "media track" without setting
>> up new receiver objects per simulcast stream.
>>
>> This leads to a few questions:
>> 1) With our current model, despite having a list of codecs, do we only
>> latch to one codec and if the sender changes codecs would that require a
>> new "receiver" object if the codec changed?
>> 2) If the current model does allow switching between codecs on the same
>> SSRC without a new receiver object, can we make it work for SVC codecs too
>> and not just non-SVC codecs?
>>
>> -Robin
>>
>>   Roman Shpount <rshpount@turbobridge.com>
>>  May 12, 2014 at 12:11 AM
>> One thing I like to mention is that we do need an ability to receive
>> multiple codecs and switch between them based on the payload type to be
>> interoperable with current VoIP. To interop with current WebRTC we need to
>> support ability to switch between OPUS, RFC 4733 tones and comfort noise
>> based on the payload type. I am also familiar with quite a few system which
>> switch the transmitting audio codec without any signaling notification to
>> the remote side, such as international VoIP carriers that switch between
>> G.729 and G.711 within the same call with no signaling.
>>
>> _____________
>> Roman Shpount
>>
>>
>>
>>   Robin Raymond <robin@hookflash.com>
>>  May 11, 2014 at 8:24 PM
>>
>> Currently the receiver has a payload type for each codec. This
>> configuration allows the receiver can receiver with any codec on the same
>> SSRC and allow auto-switching between them. This ability is true so long as
>> the codec is not SVC enabled because the layering information was defined
>> within the context of the receiver object itself and not within the context
>> of receiver's codec.
>>
>> If the receiver object is only supposed to work with a particular codec
>> only then why are we supplying the receiver object with a set of codecs and
>> payload type for each codec? So to me, that suggests the receiver object
>> CAN switch between codecs. If does allow this behaviour then why doesn't it
>> then work too for SVC codecs but only non SVC codecs?
>>
>> To me this implies we should do one of the following:
>> 1) make the sender / receiver only have one codec and that's it. Want to
>> receive or send with a different codec? Use a different sender / receiver
>> object. -OR-
>> 2) make the sender / receiver work for all codecs equally by defining the
>> layering within the context of the SVC codec and not within the context of
>> the receiver
>>
>> I'm completely fine with either scenario "1" or "2" but I don't like the
>> way it kind-of half works now. "2" is more work for ORTC implementors but
>> it allows for easier management of the receiving streams for an application
>> developer.
>>
>> -Robin
>>
>>
>