Re: Use case - alice multiway video sending using SVC and changing codecs

Agreed, it typically applies to an SFU but this scenario plays out if 
it's done locally or via an SFU. I talked about it as a local scenario 
for the sake of point #1 below but point #2 applies to both local and to 
SFU scenarios.

Point #1 - Don't re-encode the same information the same way if you can 
avoid it (optimize implicitly if you can)

When the local endpoint acts as a broadcaster, I realized we have no way 
to broadcast the same encoded stream to multiple people. Each sent 
stream requires it's own "sender" object even if they are all encoding 
identically. I'm suggesting that it would be wise to recommend 
optimizing that so the encoder will only encode once and not multiple 
times over. This recommendation applies if SVC codecs are involved or 
not. I'm NOT suggesting we expand the API to do anything here. I'm 
saying "it would be good" if an engine could figure that out and not 
re-encode again and again the same information the same way. I do not 
believe this must be a mandatory to implement this optimization but as 
soft recommendation only.

Point #2 - Let the receiver object switching between codecs work for 
both non-SVC codecs and for SVC codecs

The "receiver" object is already capable switching between more than one 
simulcast stream in a single "receiver" object. The "receiver" object 
can also switch automatically between codecs on the same SSRC too (so 
long as the codecs are non SVC enabled codecs). However, the way we have 
defined the "receiver" objects cannot switch between codecs if an SVC 
codec was enabled. Nor can the "receiver" object support SVC mixed with 
simulcast.

I'm suggesting the reason that's not possible is not some technical 
reason but only because we scoped the layering information within the 
context of the "receiver" object instead of within the context of the 
receiver's SVC codec. It's odd that we can switch codecs automatically 
so long as they aren't SVC enabled. Plus, there are real world scenarios 
where you want / need to switch from one SVC codec to another SVC codec.

If people feel making this work properly for all scenarios is too much 
for the first time out API, then perhaps we don't allow any 
auto-switching at all between any codecs nor for incoming simulcast 
streams. For an application developer to achieve this behaviour, 
independent "receiver" objects will need to be created or each simulcast 
stream and/or for each codec and the app developer will have to switch 
manually as each "receiver" object becomes active. I'm not a fan of 
leaving it half working as we will break people's code using the 
encoding parameters in the future if we fix this later.

-Robin

> Bernard Aboba <mailto:Bernard.Aboba@microsoft.com>
> May 11, 2014 at 7:19 PM
> Robin said:
>
> "1) Alice (who supports "codec1", "codec2") wants to send video to Bob,
> Charles and Debbie (who all support "codec1" and "codec2")
> 2) Alice decides to SVC encode the information once using "codec1" and
> send that encoded stream to Bob, Charles and Debbie (but since each
> receiving party aren't all equal bandwidth some will receive more layers
> than others)."
>
> [BA] Is SVC being used here in order to save encoding resources? 
> Typically we see SVC used in scenarios involving a "Scalable 
> Forwarding Unit" which forwards/drops SVC streams going to multiple 
> participants.
>
> Where Alice is sending the same layered bitstream to multiple 
> participants, it would could reuse the SVC encoding, but it would need 
> to create separate RTP streams to Bob, Charles and Debbie.
>
> So in this scenario, the savings is in creating a single layered 
> bitstream that can be adapted for the different bandwidths of each 
> participant (e.g. Alice might send all the streams to Bob, only the 
> base layer to Charles and some subset of layers to Debbie).
>
> Robin Raymond <mailto:robin@hookflash.com>
> May 11, 2014 at 11:44 AM
>
> Scenario:
> 1) Alice (who supports "codec1", "codec2") wants to send video to Bob, 
> Charles and Debbie (who all support "codec1" and "codec2")
> 2) Alice decides to SVC encode the information once using "codec1" and 
> send that encoded stream to Bob, Charles and Debbie (but since each 
> receiving party aren't all equal bandwidth some will receive more 
> layers than others).
> 3) Edward wants to receive video from Alice but only supports "codec2" 
> with SVC
> 4) Alice switches to "codec2" SVC and sends to Bob, Charles, Debbie 
> and Edward
>
>
> Currently, it would have to be setup as follows:
> 1) Alice would need to create a "RtpSender" object for each party 
> (which would have an "RtpReceiver" object)
> 2) Alice would have to configure each and every RtpSender's SVC 
> identically
> 3) When switching codecs, Alice would need to set up a brand new 
> RtpSender object per receiver and each receiver would need to create a 
> brand new RtpReceiver object when the codec was switched
>
>
> Obviously, we can argue if that use case should be supported or not 
> but this is a realistic use case I want/need ORTC be able to do. Plus, 
> since ORTC is supposed to be able to support simulcast and SVC (and 
> SVC is all about multi-party video) this doesn't seem like a far 
> fetched scenario.
>
>
> So might I suggest a couple of things to make this a LOT easier:
>
> a) make the SVC layering information as part of the codec's usage 
> definition on the receiver. This would allow switching between codecs 
> with the same SSRC for the receiver without having to set up brand new 
> receiver object every time a codec is changed (fyi - we already 
> support being able to switch codecs on the receiver where no SVC codec 
> is involved but it currently that doesn't appear possible when an SVC 
> capable codecs is used).
>
> b) to avoid SVC re-encoding again and against per sender when the 
> encoding mechanism is identical, we recommend media engines 
> auto-detect the same encoding data will be output from multiple 
> "RtpSender" objects; alternatively, we could avoid the auto-detection 
> of identical encodings by way of allowing a "RtpSender" to send via 
> more than one RtpIceTransport / RtpDtlsTransport if we want more of an 
> explicit definition than an implicit one.
>
>
> -Robin
>