Re: replaceTrack proposal

On 4 Sep 2014 04:33, "Peter Thatcher" <pthatcher@google.com> wrote:
>
> I like the idea of being able to change the track that feeds into the
> RtpSender.  Overall, I like the proposal.  But a few questions:
>
> 1.  Why not just setTrack instead of replaceTrack?  I don't mean to
> bikeshed, but we have lots of setters in the code, and no "replacers"
> in the API.
>
> 2.  Why would it matter at all what MediaStream the track is in?  I
> don't see why it would matter.  And for that matter, when would you
> have two video tracks in a MediaStream in the first place?  What does
> that even mean?

We have several applications where a sender sends multiple video streams to
a receiver. They come from different cameras and are multiplexed into a
single peer connection. This makes sense because it allows the peer
connection to manage bandwidth use across streams. Also, those streams are
identified so they can be rendered into different video elements. It would
be bass is that identification got lost during transmission. Hopefully this
is not a feature that will get lost with the move to ortc.

Regards,
Silvia.

> On Wed, Sep 3, 2014 at 7:55 AM, Jan-Ivar Bruaroey <jib@mozilla.com> wrote:
> > People want to switch seamlessly between the front and back camera on a
> > mobile device in mid-call.
> >
> > This requires somehow switching out the source of a video being sent
over a
> > peerConnection without renegotiation.
> >
> > We're trying out a solution to this that recently landed in Firefox
Nightly,
> > and we'd like to propose the API for it.
> >
> > It consists of a replaceTrack method to the recently discussed
RtpSender:
> > [1]
> >
> > interface RTCRtpSender {
> >   readonly attribute MediaStreamTrack track;
> >
> >   void replaceTrack(MediaStreamTrack withTrack,
> >                     VoidFunction successCallback,
> >                     RTCPeerConnectionErrorCallback failureCallback);
> > };
> >
> > The withTrack argument SHOULD be a track in the same mediaStream as
> > RtpSender.track, though we've had to relax this requirement until
Firefox
> > supports more than one track of the same type per mediaSteam, and the
> > restriction may turn out to be unnecessary. [2]
> >
> > The method is asynchronous (though so far we've not needed it to be).
> >
> > If the method succeeds then withTrack is now being transmitted over the
> > peerConnection in place of the previous RtpSender.track, and
RtpSender.track
> > now points to withTrack. No changes are made to the mediaStream.
> >
> > If the method fails then the existing RtpSender.track has not been
replaced.
> >
> > Example:
> >
> > function switchToSendingBackCamera(sender) {
> > mediaDevices.getUserMedia({video:{facingMode:{exact:"environment"}}},
> > function(stream) {
> >     sender.replaceTrack(stream.getVideoTracks()[0], function() {
> >       console.log("switched to back camera");
> >     }, failed);
> >   }, failed);
> > });
> >
> > Real test page:
http://mozilla.github.io/webrtc-landing/pc_test_swap.html
> > [3][4]
> >
> > .: Jan-Ivar :.
> >
> > [1]
> >
https://www.w3.org/2011/04/webrtc/wiki/images/6/63/WebRTC_RTCSender-Receiver.pdf
> >
> > [2] This works because sync happens receiver-side and timestamps are
tied to
> > capture.
> >     While streams == synchronization contexts, by replacing a track at
the
> > sender,
> >     we're making it part of the context at the receiver, so should we
> > require it on
> >     thethe sender end for symmetry?
> >
> > [3] The real test page contains antics to work on systems that are
limited
> > to one
> > open camera at a time. It calls replaceTrack twice, first to an interim
> > black
> >     fake video.
> >
> > [4] Last night's Nightly is needed to flip between front and back on
> > android.
> >
> >
>