Semantics question for onaddstream() callback

Another API question triggered by reading ROAP:

What semantics do we attach to onaddstream()?

The current spec says that the addstream event is "A new stream has been 
added to the remoteStreams array", but not when that should happen.

So ROAP may offer more information, but turns out to be a bit confusing.

The sequence of calls in section 5.3 shows (with some commentary added):

    OffererJS->OffererUA: peer=new PeerConnection();

    OffererJS->OffererUA: peer->addStream();
    OffererUA->OffererJS: sendSignalingChannel();
    OffererJS->AnswererJS: {"type":"OFFER", "sdp":"..."}
    AnswererJS->AnswererUA: peer=new PeerConnection();
    AnswererJS->AnswererUA: peer->processSignalingMessage();
               ----- at this point AnswererUA knows about the 
Offerer->Answerer proposed stream
    AnswererUA->AnswererJS: onconnecting();

    AnswererUA->OffererUA: ICE starts checking

    note right of AnswererUA: User decides it is OK to send video
    AnswererJS->AnswererUA: peer->addStream();
    AnswererUA->OffererUA: Media

    AnswererUA->AnswererJS: sendSignalingChannel();
    AnswererJS->OffererJS: {"type":"ANSWER","sdp":"..."}
    OffererJS->OffererUA: peer->processSignalingMessage();
             ---- at this point OffererUA knows that Offerer->Answerer 
stream is accepted
             ---- at this point OffererUA knows about Answerer->Offerer 
stream
    OffererUA->OffererJS: onaddstream(); <---- on the A->O stream
    OffererUA->AnswererUA: Media

    AnswererUA->OffererUA: ICE Completes
             ---- this is the earliest time media packets can actually flow
    AnswererUA->AnswererJS: onopen();
    OffererUA->OffererJS: onopen();

    OffererUA->OffererJS: sendSignalingChannel();
    OffererJS->AnswererJS: {"type":"OK" }
    AnswererJS->AnswererUA: peer->processSignalingMessage();
            ---- at this point AnswererUA knows that Answerer->Offerer 
stream is accepted
    AnswererUA->AnswererJS: onaddstream(); <----- on the O->A stream

There's an oddity here in onaddstream(): This call presumably tells the 
AnswererJS that there is an incoming video channel that will need to be 
connected to some display surface.
Now, the information needed to know this is sent in the Offer (based on 
data in the OffererJS' addStream() call), right at the top.

So in theory, the callback could be triggered at the same time as 
onconnecting(). It may also contain information that could be useful to 
know before the Answerer does his addStream() call, so there's an 
advantage to knowing that info at that time.

Now, a competing theory is that you shouldn't call onaddstream() before 
you actually see media packets. But in that case, the 
OffererUA->OffererJS: onaddstream() doesn't make sense; you can't see 
media packets until ICE has completed, and that doesn't happen until two 
lines later.

I suggest that we pick one of two semantics for onaddstream():

1) We know that this stream is expected. The callback occurs as soon as 
we have the OFFER or ANSWER containing information about this stream. 
Move the last onaddstream() up to just after onconnecting().

2) We know that media is arriving on the stream. The callback occurs as 
soon as the first media packet arrives. Move the first onaddstream() 
down below the "ICE Completes" line


My order of preference is 1) followed by 2). But I think we need to pick 
one and only one, and make sure the API spec says exactly that.

                Harald

Received on Monday, 7 November 2011 15:15:52 UTC