RtpSender/RtpReceiver and simulcast

In looking at the proposed RTCRtpSender/RTCRtpReceiver split, I had
several questions relating to the handling of encoding parameters
(e.g. resolution, framerate, quality, etc.).

My first question relates to the model for handling simulcast. 

Here is the relevant excerpt on encoding parameters 
(from http://lists.w3.org/Archives/Public/public-orca/2014Jan/0000.html):

  // Can specify multiple multiple layers or "encodings",
  // such as for simulcast, RTX, FEC, etc in the future.
  sequence<RTCRtpEncodingParameters> encodings;

  // TODO: RTP header extensions, "appID", ...

dictionary RTCRtpEncodingParameters {
  unsigned int ssrc;

  // TODO: Things to control various layers, simulcast, rtx, etc.
  // For now, let's just recognize that we need more than one
  // and we need more than the SSRC.  So let's make a dictionary,
  // and figure out the details later.
}

Looking at the above, it appears that there are two potential 
ways to handle simulcast.   One way is to utilize "application
layer simulcast" where there are multiple RTCRtpSender
objects, each sending a single stream, each of which 
utilizes its own ssrc and set of encoding parameters.  

Another way is to use "native" simulcast where a single 
RTCRtpSender object is configured to send multiple streams 
using a sequence of RTCRtpEncodingParameters. 

In "Application layer simulcast", several 
RTCRtpSender objects would be configured, each sending
a different resolution.  A single RTCRtpReceiver object
would be configured, which would be prepared to receive 
any of the potential resolutions sent by the mixer. 
In this approach, my assumption is that a "cloned"
MediaStreamTrack would be provided as input to each of the
RTCRtpSender objects. 

In "Native simulcast", a single RTCRtpSender
object would be configured with a sequence of 
RTCRtpEncodingParameters so as to send multiple streams. 
There would also be a single RTCRtpReceiver object as
before. 

In either approach the streams to be sent could be
configured to each have their own unique SSRC. 

One concern for either approach is coordination of the 
resolutions to be sent.  For example, it makes no sense
for the browser to send the same resolution on multiple 
streams.   

Also, to avoid disconcerting the user, it would typically
be desirable for the resolutions sent or received to have
a consistent aspect ratio. 

To provide a concrete example, let us say an application
wishes to offer to simultaneously send resolutions of
640 by 360 (on PT 97) and 320 by 180 (on PT 98), as well
as to receive either resolution (on PT 97).

How would we configure the RTCRtpSender and RTCRtpReceiver
objects to do this? 

For those willing to countenance the vulgarities of SDP
below is an example O/A exchange conforming to the spirit
of "unified plan". 

Warning:  This SDP O/A Exchange is rated (R)estricted, due to
the presence of adult material.  Children Under 17 Require
Accompanying Parent or Adult Guardian. 

Offer: 

v=0
o=alice 2362969037 2362969040 IN IP4 192.0.2.156
s=Offer from Simulcast Enabled Browser to Mixer
t=0 0
c=IN IP4 192.0.2.156
b=AS:665
a=group:SCR S1 S2
m=video 49300 RTP/AVP 97
b=AS:520
a=rtpmap:97 H264/90000
a=fmtp:97 profile-level-id=42c01e
a=imageattr:97 send [x=640,y=360] recv [x=640,y=360][x=320,y=180]
a=mid:S1
m=video 49302 RTP/AVP 98
a=sendonly
a=rtpmap:98 H264/90000
a=fmtp:98 profile-level-id=42c01e
a=imageattr:98 send [x=320,y=180]
a=mid:S2

Answer: 

v=0
o=server 823479283 1209384938 IN IP4 192.0.2.2
s=Answer to Simulcast Enabled Client
t=0 0
c=IN IP4 192.0.2.43
b=AS:665
a=group:SCR S1 S2
m=video 48000 RTP/AVP 97
b=AS:520
a=rtpmap:97 H264/90000
a=fmtp:97 profile-level-id=42c01e
a=imageattr:97 send [x=640,y=360][x=320,y=180] recv [x=640,y=360]

Received on Friday, 10 January 2014 02:09:58 UTC