Re: A proposal for splitting RTCTrack into RtpSender and RtpReceiver.

Separation of send and receive objects seems like a powerful idea. 

Off the bat, I think it will help with simulcast which is inherently asymmetric, because while a browser may send multiple encodings, it will typically only receive one at a time from the mixer. As a result, modeling simulcast with distinct send and receive objects is pretty natural. A browser supporting simulcast might have multiple send objects but only one receive object (whose parameters might change on the fly without signaling). 

It also seems like it should help with layered coding, where send and receive parameters as well as protection settings may differ substantially between send and receive but that aspect will need more exploration.

> On Jan 6, 2014, at 6:05 PM, "Erik Lagerway" <elagerway@gmail.com> wrote:
> 
> I personally like the concepts here and the design. It's great to see Google become active in the ORCA CG.
> 
> Thanks for the contribution!
> 
> Erik
> 
>> On Jan 6, 2014, at 3:41 PM, Peter Thatcher <pthatcher@google.com> wrote:
>> 
>> Some of the discussions in the recent W3C WebRTC WG have centered around the idea of having "rtp send doohickeys" and "rtp receive doohickeys", which would represent objects that control sending and receiving a single media track, respectively.  I think this would be a good direction for the ORTC API to take, and I think would improve upon the ORTC API greatly.   You can think of this as splitting the "RTCTrack" into two classes: a "sender" class and a "receiver" class, refining their methods and attributes.  
>> 
>> I propose an API something like the following.  I'm using the names "RtpSender" and  "RtpReceiver" which Justin Uberti proposed to the W3C.
>> 
>> // An RTCRtpSender sends one MediaStreamTrack 
>> // over one RTCConnection.
>> // If you want to send a whole MediaStream with multiple tracks,
>> // Create multiple RtpSenders.
>> [Constructor(MediaStreamTrack track, ​RTC​Connection​ ​transport​)]
>> interface RTCRtpSender {
>>  readonly attribute MediaStreamTrack ​track;
>>  readonly ​attribute RTCConnection ​transport;
>> 
>>  // The way media is sent is controlled by the given "parameters".
>>  // The sender starts sending when send() is called, 
>>  // And stops sending when stop() is called.
>>  // send() may be called more than once to change parameters.
>>  // stop() is final, just like MediaStreamTrack.
>>  void send(RTCRtpParameters​ parameters);
>> ​ ​ void stop();
>> 
>>  // NOTE: We can't have a readonly attribute because apparently
>>  // In WebIDL, "readonly" doesn't make the returned object 
>>  // readonly, just the reference. 
>>  RTCRtpParameters parameters();
>> 
>>  // Instead of a top-level getRtcCapabilities and getRtcCodecs(),
>>  // We have a static method on the RtpSender and RtpReceiver
>>  // (send and receive capabilities might be different).
>> ​  ​static RTCRtpCapabilities getCapabilities();
>>  // While JS can construct and negotiate parameters just based on 
>>  // the capabilities, doing so is usually a pain, so we provide two 
>>  // convenience methods to help with the process.  The first 
>>  // creates parameters based on capabilities and the second filters 
>>  // already created paremters based on capabilities.  This allows 
>>  // for a lot of flexibility of which side does the neogitation and 
>>  // when.(We're not confined to an offer/answer model).
>> ​  static RtpParameters createParameters(
>>    MediaStreamTrack ​track
>>    optional ​RTCRtpCapabilities capabilities);
>> ​  ​static RTCRtpParameters filterParameters(
>>    RTCRtpParameters parameters,
>> ​    optional RTCRtpCapabilities capabilities);​
>> }​
>> 
>> // An RTCRtpReceiver receives one MediaStreamTrack over one 
>> // RTCConnection.  If you want to receive a whole MediaStream with 
>> // multiple tracks, create multiple RtpReceivers.
>> [Constructor(optional ​RTC​Connection​ ​transport)]
>> interface RTCRtpReceiver {
>> ​  // The media coming from the transport is available from the 
>>  // track.​  The track is null until receive() is called, and once  
>>  // it is set to non-null, will not changed.
>>  readonly attribute MediaStreamTrack? track;
>>  readonly attribute ​RTC​Connection transport;
>> 
>>  // The way media is received is controlled by the given 
>>  // "parameters".  The receiver starts receiving when receive() is 
>>  // called, and stops sending when stop() is called.
>>  // receive() may be called more than once to change parameters.
>>  // stop() is final, and the receive can never receive again.
>>  void receive(RTCRtpParameters​ parameters);
>> ​ ​ void stop();
>> 
>>  // NOTE: We can't have a readonly attribute because apparently
>>  // In WebIDL, "readonly" doesn't make the returned object 
>>  RTCRtpParameters parameters();
>> 
>>  // Instead of a top-level getRtcCapabilities and getRtcCodecs(),
>>  // We have a static method on the RtpSender and RtpReceiver
>>  // (send and receive capabilities might be different).
>> ​  ​static RTCRtpCapabilities getCapabilities();
>>  // While JS can construct and negotiate parameters just based on 
>>  // the capabilities, doing so is usually a pain, so we provide two 
>>  // convenience methods to help with the process.  The first 
>>  // creates parameters based on capabilities and the second filters 
>>  // already created paremters based on capabilities.  This allows 
>>  // for a lot of flexibility of which side does the neogitation and 
>>  // when. (We're not confined to an offer/answer model).
>> ​  static RtpParameters createParameters(
>>    DOMString kind,  // "audio" or "video"
>>    ​optional RTCRtpCapabilities capabilities);
>> ​  ​static RTCRtpParameters filterParameters(
>>    RTCRtpParameters parameters,
>> ​    optional RTCRtpCapabilities capabilities);​
>> }
>> 
>> // The RTCRtpParameters contains much of what currently exists as / 
>> // attributes of RTCTrack, but as a dictionary, it's much easier to 
>> // serialize/deserialize for signalling, and pass into and out from 
>> // convenience methods for neogitation.  Finally, it allows changing 
>> // of parameters in an atomic way.
>> dictionary RTCRtpParameters {
>>  sequence<RTCRtpCodec> codecs;
>> 
>>  // Can specify multiple multiple layers or "encodings",   
>>  // such as for simulcast, RTX, FEC, etc in the future.  
>>  sequence<RTCRtpEncodingParameters> encodings;
>> 
>>  // TODO: RTP header extensions, "appID", ...
>> }
>> 
>> dictionary RTCRtpEncodingParameters {
>>  unsigned int ssrc;
>> 
>>  // TODO: Things to control various layers, simulcast, rtx, etc.
>>  // For now, let's just recognize that we need more than one 
>>  // and we need more than the SSRC.  So let's make a dictionary,
>>  // and figure out the details later.
>> }
>> 
>> 
>> Finally, here's an example of how it could be used:
>> 
>> // Assume we already have a way to signal, a transport 
>> // (RTCConnection), and audio and video tracks. This is an example 
>> // of  how to offer them  and get back an answer with audio and 
>> // video tracks, and begin sending and receiving them.
>> function initiate(signaller, transport, audioTrack, videoTrack) {
>>  var audioSender = new RTCRtpSender(audioTrack, transport);
>>  var videoSender = new RTCRtpSender(videoTrack, transport);
>>  var audioReceiver = new RTCRtpReceiver(transport);
>>  var videoReceiver = new RTCRtpReceiver(transport);
>> 
>>  var sendAudioParams = RTCRtpSender.createParameters(audioTrack);
>>  var sendVideoParams = RTCRtpSender.createParameters(videoTrack);
>>  signaller.offerTracks({
>>    // The initiator offers parameters it wants to send with, 
>>    // and the capabilities it has for receiving. 
>>    "rtpCaps": RTCRtpReceiver.getCapabilities(),
>>    "audio": sendAudioParams,
>>    "video": sendVideoParams
>>  }, function(answer) {
>>    // The responder answers with parameters it wants to send with
>>    // and the capabilities it has for receiving. 
>>    audioSendParams = RTCRtpSender.filterParameters(
>>       sendAudioParams, answer.rtpCaps);
>>    videoSendParams = RTCRtpSender.filterParameters(
>>       sendVideoParams, answer.rtpCaps
>>    var audioRecvParams = RTCRtpSender.filterParameters(
>>       answer.audio);
>>    var videoRecvParams = RTCRtpSender.filterParameters(
>>       answer.video);
>> 
>>    audioSender.send(audioSendParams);
>>    videoSender.send(videoSendParams)
>>    audioReceiver.receive(audioRecvParams);
>>    videoReceiver.receive(videoRecvParams);
>> 
>>    // Now we can render/play 
>>    // audioReceiver.track and videoReceiver.track.
>>  });
>> }
>> 
>> // Assume we already have a way to signal, a transport 
>> // (RTCConnection), and audio and video tracks. This is an example 
>> // of how to answer an offer with audio and video tracks, and begin 
>> // sending and receiving them.
>> function accept(
>>  signaller, remote, transport, audioTrack, videoTrack) {
>>  var audioSender = new RTCRtpSender(audioTrack, transport);
>>  var videoSender = new RTCRtpSender(videoTrack, transport);
>>  var audioReceiver = new RTCRtpReceiver(transport);
>>  var videoReceiver = new RTCRtpReceiver(transport);
>> 
>>  var audioSendParams = RTCRtpSender.createParameters(
>>    audioTrack, remote.rtpCaps);
>>  var videoSendParams = RTCRtpSender.createParameters(
>>    videoTrack, remote.rtpCaps);
>>  var audioRecvParams = RTCRtpSender.filterParameters(
>>     remote.audio);
>>  var videoRecvParams = RTCRtpSender.filterParameters(
>>     remote.video);
>> 
>>  audioSender.send(audioSendParams);
>>  videoSender.send(videoSendParams)
>>  audioReceiver.receive(audioRecvParams);
>>  videoReceiver.receive(videoRecvParams);
>> 
>>  signaller.answerTracks({
>>    "rtpCaps": RTCRtpReceiver.getCapabilities(),
>>    "audio": audioSender.parameters(),
>>    "video": videoSender.parameters()
>>  });
>> 
>>  // Now we can render/play 
>>  // audioReceiver.track and videoReceiver.track.
>> }
>

Received on Tuesday, 7 January 2014 04:58:11 UTC