Re: Update of RTCRtpSender "doohickey" proposal

On Mon, Apr 28, 2014 at 9:33 AM, Peter Thatcher <pthatcher@google.com>wrote:

> I like the addition of the streamId to addTrack and stream to
> RemoteTrackEvent.  It fixes the issues discussed before about tracks and
> streams in a simple way.  Just a few questions about it:
>
> 1.  Does the streamId need to be the ID of an already existing stream?  Or
> can it be an arbitrary value chosen by the JS?
>

I thought about it a bit, and I don't really see an issue with making it
arbitrary. This will put the track to be in the specified MS at the other
side, but I don't see a need to force the tracks to be bagged up on the
send side.

>
> 2.  Would it make sense to allow streamId to be optional, such that it's
> possible to simply call addTrack(track) and have it just work?  Or are the
> dragons with that?
>

I think it would be fine to have it optional. The question becomes what the
receiver gets in this case - either a null stream param in the
RemoteTrackEvent, or an auto-generated stream. If null, the receiver will
need to create a new MediaStream in order to play out the track, which
makes life a bit harder for simple apps, but is probably the most
straightforward solution for advanced apps.

>
>
> And I think the RTCRtpEncodingParams on the knobs inside of it are
> potentially a very large conversation. I think the
> addTrack/RtpSender/RtpReceiver/DtlsTransport is good enough by itself that
> it shouldn't be blocked by getting the knobs inside of RTCRtpEncodingParams
> perfect.
>
>
>
> On Mon, Apr 28, 2014 at 8:58 AM, Justin Uberti <juberti@google.com> wrote:
>
>> Update of the proposal from earlier this year, addressing the feedback
>> from the list. Mainly, the issues with AddStream and AddTrack have been
>> resolved by adding an explicit |stream| parameter to AddTrack.
>>
>>
>> ------------------------------------------------------------------
>>
>>
>> I suggest we call the SendDoohickeys RTCRtpSenders and the corresponding
>> receive-side objects RTCRtpReceivers. These objects allow control of how a
>> MediaStreamTrack is encoded and sent on the wire, including "hold" state,
>> prioritization, and multiple encodings (e.g. simulcast).
>>
>> You get a RTCRtpSender as a return value from AddTrack (which replaces
>> AddStream). You get a RTCRtpReceiver as an argument to the new onaddtrack
>> callback (which replaces onaddstream). The actual track object can be
>> obtained as a property from the RTCRtpReceiver (see below).
>>
>> For getting access to ICE/DTLS info, both RTCRtpSenders and
>> RTCRtpReceivers can also have a reference to a RTCDtlsTransport object,
>> which will have its own state variables, including the
>> RTCIceConnectionState of that particular transport, and the
>> .remoteCertificates for the DTLS connection. This allows applications to
>> monitor the state of individual transports, as well as inspect the
>> certificates for individual transports.
>>
>> The actual interface is as follows:
>>
>> // used with RTCRtpSender
>>
>> interface RTCDtlsTransport {
>>
>>  attribute RTCIceConnectionState state;
>>
>>  sequence<ArrayBuffer> getRemoteCertificates();
>>
>>  //... more stuff as needed
>>
>> };
>>
>> // used with RTCRtpSender
>>
>> interface RTCRtpEncodingParams {
>>
>>     double               priority = 1.0;  // relative priority of this
>> encoding
>>
>>     unsigned int       maxBitrate = null;  // maximum bits to use for
>> this encoding
>>
>>     boolean             active;  // sending or "paused/onhold"
>>
>> };
>>
>> // the "send" doohickey
>>
>> interface RTCRtpSender {
>>
>>  readonly attribute MediaStreamTrack track;
>>
>>  readonly attribute RTCDtlsTransport transport;
>>
>>  // 1-N encodings; in the future, N can be > 1, for simulcast or layered
>> coding
>>
>>  // Each EncodingParams specifies the details of what to send (e.g.
>> bitrate)
>>
>>  sequence<RTCRtpEncodingParams> getEncodings();
>>
>> };
>>
>> // the "receive" doohickey
>>
>> interface RTCRtpReceiver {
>>
>>   readonly attribute RTCDtlsTransport transport;
>>
>>   readonly attribute MediaStreamTrack track;
>>
>>   // more stuff as needed
>>
>> };
>>
>> // parameter to the onaddtrack event
>>
>> interface RemoteTrackEvent : Event {
>>
>>  readonly attribute RtpReceiver receiver;
>>
>>  readonly attribute MediaStreamTrack track;
>>
>>  readonly attribute MediaStream stream;
>>
>> };
>>
>> partial interface RTCPeerConnection {
>>
>>  // because a track can be part of multiple streams, the id parameter
>>
>>   // indicates which particular stream should be referenced in signaling
>>
>>  RtpSender addTrack(track, streamId);  // replaces addStream
>>
>>
>>
>>  void removeTrack(RtpSender);  // replaces removeStream
>>
>>  sequence<RtpSender> getSenders();
>>
>>  sequence<RtpReceiver> getReceivers();
>>
>>  EventHandler onaddtrack;  // replaces onaddstream; event object is
>> RemoteTrackEvent.
>>
>>                                         // note that onremovestream is
>> not needed, since tracks are 'removed'
>>
>>                                         // simply by progressing to the
>> ENDED state
>>
>> };
>>
>> For backcompat, addStream, removeStream, getLocalStreams,
>> getRemoteStreams, and onaddstream can be trivially polyfilled in JS, so
>> there is minimal impact on current applications.
>>
>> All together, the pipeline looks like this:
>>
>> Source ---> MST ---> RtpSender ---> DtlsTransport ---> (The Internet)
>> ---> DtlsTransport ---> RtpReceiver ---> MST ---> <video/>
>>
>> Each RtpSender/Receiver references a single MST, although a single
>> RtpSender/Receiver can send/receive multiple encodings (e.g. simulcast).
>>
>> There are N RtpSenders/Receivers per DtlsTransport; N is controlled by
>> the policy specified for BUNDLE.
>>
>>
>