Re: replaceTrack proposal

On Wed, Sep 3, 2014 at 3:49 PM, Silvia Pfeiffer
<silviapfeiffer1@gmail.com> wrote:
>
> On 4 Sep 2014 04:33, "Peter Thatcher" <pthatcher@google.com> wrote:
>>
>> I like the idea of being able to change the track that feeds into the
>> RtpSender.  Overall, I like the proposal.  But a few questions:
>>
>> 1.  Why not just setTrack instead of replaceTrack?  I don't mean to
>> bikeshed, but we have lots of setters in the code, and no "replacers"
>> in the API.
>>
>> 2.  Why would it matter at all what MediaStream the track is in?  I
>> don't see why it would matter.  And for that matter, when would you
>> have two video tracks in a MediaStream in the first place?  What does
>> that even mean?
>
> We have several applications where a sender sends multiple video streams to
> a receiver. They come from different cameras and are multiplexed into a
> single peer connection. This makes sense because it allows the peer
> connection to manage bandwidth use across streams. Also, those streams are
> identified so they can be rendered into different video elements.
> It would be bass is that identification got lost during transmission. Hopefully this
> is not a feature that will get lost with the move to ortc.


If you want multiple video streams synchronized, then sticking them in
one MediaStream would make sense.  I hadn't considered that.  But you
get multiplexing over one peer connection with bandwidth managed
across those streams regardless of whether they are in one MediaStream
or many.


>
> Regards,
> Silvia.
>
>> On Wed, Sep 3, 2014 at 7:55 AM, Jan-Ivar Bruaroey <jib@mozilla.com> wrote:
>> > People want to switch seamlessly between the front and back camera on a
>> > mobile device in mid-call.
>> >
>> > This requires somehow switching out the source of a video being sent
>> > over a
>> > peerConnection without renegotiation.
>> >
>> > We're trying out a solution to this that recently landed in Firefox
>> > Nightly,
>> > and we'd like to propose the API for it.
>> >
>> > It consists of a replaceTrack method to the recently discussed
>> > RtpSender:
>> > [1]
>> >
>> > interface RTCRtpSender {
>> >   readonly attribute MediaStreamTrack track;
>> >
>> >   void replaceTrack(MediaStreamTrack withTrack,
>> >                     VoidFunction successCallback,
>> >                     RTCPeerConnectionErrorCallback failureCallback);
>> > };
>> >
>> > The withTrack argument SHOULD be a track in the same mediaStream as
>> > RtpSender.track, though we've had to relax this requirement until
>> > Firefox
>> > supports more than one track of the same type per mediaSteam, and the
>> > restriction may turn out to be unnecessary. [2]
>> >
>> > The method is asynchronous (though so far we've not needed it to be).
>> >
>> > If the method succeeds then withTrack is now being transmitted over the
>> > peerConnection in place of the previous RtpSender.track, and
>> > RtpSender.track
>> > now points to withTrack. No changes are made to the mediaStream.
>> >
>> > If the method fails then the existing RtpSender.track has not been
>> > replaced.
>> >
>> > Example:
>> >
>> > function switchToSendingBackCamera(sender) {
>> > mediaDevices.getUserMedia({video:{facingMode:{exact:"environment"}}},
>> > function(stream) {
>> >     sender.replaceTrack(stream.getVideoTracks()[0], function() {
>> >       console.log("switched to back camera");
>> >     }, failed);
>> >   }, failed);
>> > });
>> >
>> > Real test page:
>> > http://mozilla.github.io/webrtc-landing/pc_test_swap.html
>> > [3][4]
>> >
>> > .: Jan-Ivar :.
>> >
>> > [1]
>> >
>> > https://www.w3.org/2011/04/webrtc/wiki/images/6/63/WebRTC_RTCSender-Receiver.pdf
>> >
>> > [2] This works because sync happens receiver-side and timestamps are
>> > tied to
>> > capture.
>> >     While streams == synchronization contexts, by replacing a track at
>> > the
>> > sender,
>> >     we're making it part of the context at the receiver, so should we
>> > require it on
>> >     thethe sender end for symmetry?
>> >
>> > [3] The real test page contains antics to work on systems that are
>> > limited
>> > to one
>> > open camera at a time. It calls replaceTrack twice, first to an interim
>> > black
>> >     fake video.
>> >
>> > [4] Last night's Nightly is needed to flip between front and back on
>> > android.
>> >
>> >
>>

Received on Wednesday, 3 September 2014 22:58:16 UTC