Re: ReplaceTrack vs RTPSender.track = new

Hi Harald, I had some trouble following the example with my knowledge, 
and have some questions if you don't mind.

On 4/15/15 3:31 AM, Harald Alvestrand wrote:
> In mulling over this issue, I've been thinking about failure cases, 
> and about negotiated constraints.
>
> Consider this case:
>
> A negotiates with B to send video
> The agreeed capabilities for B is 320x200 max (it's a doorbell, I guess).

I'm not sure what "negotiated constraints" means. Are we talking about 
session (e.g. hardware) abilities in SDP here (I assume so since you 
mention doorbell)?

If so, then doesn't the 320x200 track need to be added first in order to 
negotiate video?

> A sends a track to B with 320x200 format.

If the track somehow wasn't added first, but is added later instead, A 
would need to renegotiate, right? Say it was an HD track, how would it 
work/fail today?

> A does ReplaceTrack with a HD track (currently also being displayed in 
> HD somewhere else).
>
> Three possibilities:
>
> - A downsamples to 320x200 as (conceptuaully) an internal function of 
> the PeerConnection

My understanding of SDP is limited (assuming we are talking about SDP). 
What does A know about B's abilities beyond the answer, which is a 
subset of the offer, which, as I understand it, would only (need to) 
cover the 320x200 track added?

Both PRs say "Things like dimensions and frame rate do not require 
negotiation." - I relied on other people when writing that, but this may 
assume things about the other end. For example, in ideal cases where the 
recipient is much like the sender (e.g. a local-loop test), we can 
replaceTrack between tracks with different dimensions and frameRates, 
and the recipient handles the change without renegotiation (works in 
Firefox: http://jsfiddle.net/oyabqd0j )! This is powerful, but a 
doorbell would probably react poorly to this, so how to detect this 
ahead of time from the SDP?

Does the doorbell wreck it for everybody?

> - A treats B's restrictions as (new) constraints on the track, and all 
> other users switch to 320x200

In replaceTrack? No, PeerConnections can't change constraints. 
Constraints is a gUM concept. To a PeerConnection, a track here is a 
hose with video coming out of it. It can clamp the hose, but otherwise 
has little control over what comes out of it. The knobs on the hose 
(applyConstraints) are for the website to mess with, not PeerConnection, 
so whatever "A" is in the example, it can't be PeerConnection.

If A, the website, wants to change camera constraints, then that's on it 
to manage (though how would it learn of B's hardware limitations in this 
case?)

> - ReplaceTrack fails

You skipped renegotiation, but I suspect our current implementation 
would neither fail nor renegotiate over this (see two up). I confess not 
having tested a doorbell.

> Replay the same case where the source of the new track is a VP8-only 
> camera, and B says it can only take H.264 (I guess it's a gateway).
>
> Three possibilities:
>
> - A transcodes the video internally to PeerConnection

What do we do today for renegotiation? The PR notes " Reasons 
negotiation may be needed include video differing in raw vs. pre-encoded 
format".

> - A inserts transcoding at the source

Again, can't alter the source AFAIK.

> - ReplaceTrack fails

We probably fire negotiationneeded instead here.

> And of course the weird one:
>
> A does ReplaceTrack, trying to replace an audio track with a video track.

This one is covered in both PRs:

  * "1. If withTrack's kind attribute differs from the kind of
    RTCRtpSender.track, then reject p with
    IncompatibleMediaStreamTrackError and abort these steps."
  * "3. If newTrack's kind attribute differs from the kind of oldTrack,
    then throw an IncompatibleMediaStreamTrackError exception and abort
    these steps."


I would love to understand the example, even if the conclusion of the 
subject is foregone, as it may have wider implications to what's 
possible with replaceTrack and what's not.

.: Jan-Ivar :.