Re: Update of Doohickey/AddTrack proposal

On 2013-12-19 19:13, Martin Thomson wrote:
> I think that you need to talk a little bit about how you plan to
> manage the difference between send and receive tracks.
>
> The onaddtrack (which I think would be even clearer as onremotetrack)
> needs a description of its arguments.  I think that those should be:
>
> interface RemoteTrackEvent : Event {
>   readonly attribute RtpReceiver receiver;
>   readonly attribute MediaStreamTrack track;
>   readonly attribute MediaStream[] streams;
> };
>
> The stream here is important.  This makes onaddstream polyfill
> actually possible.  It's otherwise impossible to learn what stream is
> being sent.  The fact that a single track can be part of multiple
> streams means that you need this to be cardinality n.

I'm less convinced we should go this way.

Say that the sending app has MediaStreamTrack A in three different local
MediaStreams. If if it now does addTrack(A), the streams attribute would
indicate that the received track belongs to three different MediaStreams
- which is not really helpful because there is no info which one(s) that
the sending app wants to send.

But that can be managed I think, but what if the sender side app now
creates yet another local MediaStream and includes track A in it, should
that cause an offer/answer to signal this?

To me it seems much more straightforward to send and receive tracks
only; the receiving app is responsible for including received tracks in
MediaStreams for recording or playout. It makes polyfill more difficult,
but I imagine that most cases can be handled by the receiving app just
creating one MediaStream which is used to collect all incoming tracks
(because in most cases it will be just one video and one audio). For
more advanced cases there would have to be some app specific signaling
(i.e. tracks A and B represents this, track C that) - or the "content
label" is introduced and used.

>
> On 18 December 2013 19:35, Justin Uberti <juberti@google.com> wrote:
>> Based on some feedback I've received, I've slightly updated my proposal
>> (originally inspired by Martin) on doohickeys and replacing AddStream with
>> AddTrack. See below:
>>
>> ------------------------------------------------------------------------
>>
>> I suggest we call the SendDoohickeys RtpSenders and the corresponding
>> receive-side objects RtpReceivers. These objects allow control of how a
>> MediaStreamTrack is encoded and sent on the wire, including "hold" state,
>> prioritization, and multiple encodings (e.g. simulcast).
>>
>> You get a RtpSender as a return value from AddTrack (which replaces
>> AddStream). You get a RtpReceiver as an argument to the new onaddtrack
>> callback (which replaces onaddstream). The actual track object can be
>> obtained as a property from the RtpReceiver (see below).
>>
>> For getting access to ICE/DTLS info, both RtpSenders and RtpReceivers can
>> also have a reference to a DtlsTransport object, which will have its own
>> state variables, including the RTCIceConnectionState of that particular
>> transport, and the .remoteCertificates for the DTLS connection. This allows
>> applications to monitor the state of individual transports, as well as
>> inspect the certificates for individual transports.
>>
>> The actual interface is as follows:
>>
>> interface DtlsTransport {
>>   attribute RTCIceConnectionState state;
>>   attribute sequence<ArrayBuffer> remoteCertificates;
>>   //... more stuff as needed
>> };
>>
>> interface RtpSender {
>>   readonly attribute MediaStreamTrack track;
>>   readonly attribute DtlsTransport transport;
>>   // various tweakable attributes
>>   attribute bool active;  // controls "am I sending RTP"
>>   attribute PriorityLevel priority;  // for relative bandwidth allocation
>>   // for multiple encodings: simulcast (audio or video), layered coding
>>   // specify the codec to use, resolution, bitrate, and any dependencies for
>> each encoding
>>   attribute sequence<EncodingParams> encodings;
>> };
>>
>> interface RtpReceiver {
>>   readonly attribute Transport transport;
>>   readonly attribute MediaStreamTrack track;
>>   // more stuff as needed
>> };
>>
>> partial interface PeerConnection {
>>   RtpSender addTrack(MST);  // replaces addStream
>>   void removeTrack(RtpSender);  // replaces removeStream
>>   readonly attribute sequence<RtpSender> senders;
>>   readonly attribute sequence<RtpReceiver> receivers;
>>   EventHandler onaddtrack;  // replaces onaddstream; onremovestream is not
>> needed
>> };
>>
>> For backcompat, addStream, removeStream, getLocalStreams, getRemoteStreams,
>> and onaddstream can be trivially polyfilled in JS, so there is minimal
>> impact on current applications.
>>
>> All together, the pipeline looks like this:
>>
>> Source ---> MST ---> RtpSender ---> Transport ---> (The Internet) --->
>> Transport ---> RtpReceiver ---> MST ---> <video/>
>>
>> Each RtpSender/Receiver references a single MST, although a single
>> RtpSender/Receiver can send/receive multiple encodings (e.g. simulcast).
>> There are N RtpSenders/Receivers per Transport; N is controlled by the
>> policy specified for BUNDLE.
>>
>

Received on Friday, 20 December 2013 11:53:17 UTC