Re: Proposal how to map JSEP to the existing API

On 02/10/2012 03:54 PM, Justin Uberti wrote:
>
>
> On Fri, Feb 10, 2012 at 5:25 AM, Adam Bergkvist
> <adam.bergkvist@ericsson.com <mailto:adam.bergkvist@ericsson.com>> wrote:
>
>     Hi
>
>     The JavaScript Session Establishment Protocol (JSEP) imposes a new
>     set of requirements on the signaling flow. One significant
>     improvement is the separation of connectivity establishment (ICE)
>     and media negotiation (SDP O-A) to allow for ICE "trickling".
>     In addition, there are new requirements for handling of session
>     descriptions (from the JSEP draft):
>     1. To know if a session description pertains to the local or remote
>     side.
>     2. To know if a session description is an offer or an answer.
>     3. To allow the offer to be specified independently of the answer.
>
>     We would like to map this new protocol to the existing API to allow
>     for the simple case to remain simple, yet powerful. Key concepts
>     that we would like to preserve include that the browser knows when a
>     new signaling message needs to be conveyed to the other peer as well
>     as how to handle incoming signaling messages. A web developer who
>     would just like to get something working should not have to learn
>     specifics about ICE and SDP and know how the signaling works and,
>     for example, when some local action requires an updated offer to be
>     created.
>
>     To support the requirements of JSEP, we propose the following
>     changes to the existing API:
>
>     For the browser to easily be able to determine whether a signaling
>     message is an offer, answer, or candidate line, a corresponding
>     keyword is added to the signaling message header (#2 above). For the
>     browser to know if an offer or answer was created locally, each
>     PeerConnection instance is assigned an internal unique identifier
>     that is also included in the signaling message header (#1 above).
>
>     To allow emitted offers and answers to be updated locally,
>     processSignalingMessage() is changed to support signaling messages
>     to be passed into the same PeerConnection instance that they
>     originated from (#3 above).
>
>     // set local description with offer
>     processSignalingMessage("SDP OFFER <id>\n<sdp>");     // id is local
>
>     // set remote description with offer
>     processSignalingMessage("SDP OFFER <id>\n<sdp>");     // id is not local
>
>     // set local description with answer
>     processSignalingMessage("SDP ANSWER <id>\n<sdp>");    // id is local
>
>     // set remote description with answer
>     processSignalingMessage("SDP ANSWER <id>\n<sdp>");    // id is not local
>
>     // process ICE candidate
>     processSignalingMessage("SDP CANDIDATE <id>\n<sdp>");
>
>
>
>     Signaling messages with offers and answers need to be produced
>     asynchronously by the browser. When modifications have been made
>     that requires an updated offer, a new signaling message is
>     automatically emitted when stable state is reached (no change).
>     Likewise, when a remote offer is passed into a PeerConnection
>     instance, an answer is automatically produced when stable state is
>     reached (no change).
>
>
> I understand the intent here, but I feel like this may be
> oversimplifying. First, I don't think the proposed JSEP API requires any
> knowledge of SDP or ICE, only basic O-A semantics; as W3C APIs go, it is
> a lot simpler than many others.

For example, the developer needs to know when to start ICE processing, 
the differences between ICE candidate messages, SDP offers and SDP 
answers, as well as when to generate new SDP offers.

> Second, I don't think an API that you
> have to use sprintf() to call is in sync with general programming
> trends; I think we want this to be an API, not a protocol handler.

I'm not proposing to force the web developer to format signaling 
messages manually. The examples above include parts of the signaling 
messages in the arguments to processSignalingMessage() that are used by 
the browser to determine what kind of message it gets.

A signaling message, emitted from PeerConnection's signalingCallback 
already has this header, and processSignalingMessage(), which inputs 
messages to PeerConnection, expects this header. I.e., the developer can 
treat the messages as opaque strings in the simple example (shown 
below). A developer who wants to manipulate a signaling message can 
ignore the header and only change the sdp part. Note that the signaling 
message is given to the application and transporting it to the other 
side is up to the application.

// sending signaling messages
new PeerConnection("", signalingChannel.send);

// receiving (and processing) signaling messages
signalingChannel.onmessage = function (evt) {
     pc.processSignalingMessage(evt.data);
};

// signalingChannnel is provided by the application

> Lastly, it also does not faithfully map all the methods/concepts of the
> proposed JSEP API, some of which are indicated below.
>
> That said, if we want to provide this sort of mapping within a JS
> library, in order to provide a higher-layer interface to people who are
> willing to trade off control for convenience, that seems entirely
> reasonable. As mentioned in the draft, more advanced protocol concepts
> could be provided in this library.

I think it's far from optimal for a brand new Web API to be dependent on 
JS libraries to be easy to use. Our audience should be web developers 
rather than JS library developers.

>     Signaling flow, connectivity establishment
>     ------------------------------__------------
>
>     A:
>     new PeerConnection()
>
>     // stable state, start gathering ICE candidates
>
>     [ signalingCallback ] "SDP CANDIDATE <id>\n<sdp>"
>     // send candidate to B
>
>     B:
>     new PeerConnection()
>     processSignalingMessage("SDP CANDIDATE <id>\n<sdp>")
>
>
> We can't start sending candidates until we've at least dispatched an
> offer; there are no m= lines or ICE credentials at this point.

This example only shows the connectivity establishment. If the 
candidates end up being dependent on an offer, that would just be a rule 
that would be enforced by the browser. For example, the first candidate 
must be included in an offer.

>     // stable state, start gathering ICE candidates
>
>     [ signalingCallback ] "SDP CANDIDATE <id>\n<sdp>"
>     // send candidate to A
>
>
> We need to be able to control which candidates are used for connectivity
> checks here.

The candidate is handed to the application. It can simply be discarded 
if it should not be used for connectivity checks (advanced usage).

>     A:
>     // start establishing connectivity
>     processSignalingMessage("SDP CANDIDATE <id>\n<sdp>")
>
>
>     Signaling flow, media negotiation
>     ------------------------------__---
>
>     A:
>     addStream()
>
>     // stable state, create offer
>     [ signalingCallback ] "SDP OFFER <id>\n<sdp>"
>
>     *** Offer may be modified and reinserted locally with
>     processSignalingMessage()
>     // send offer to B
>
>
> Can hints be supplied to control the generation of the offer?

Yes, addStream() has an optional second argument for hints.

>     B:
>     addStream()
>
>     // process incoming offer
>     processSignalingMessage("SDP OFFER <id>\n<sdp>")   // id is not local
>
>     // stable state, create answer
>
>     [ signalingCallback ] "SDP ANSWER <id>\n<sdp>"
>
>
> We don't want to generate an ANSWER here, as the callee has not yet
> consented to the call. Also, as above, can hints be supplied to control
> the generation of the answer? Can a provisional answer be generated and
> installed?

In the case of a call, the application can wait for user consent either 
before calling processSignalingMessage(offer) or before sending the 
answer to A. A (provisional) answer is installed as it's generated, but 
if we want to allow it, the answer can be replaced as described by the 
comment below.


>     *** Answer may be modified and reinserted locally with
>     processSignalingMessage()
>
>     // B has all info needed to receive
>     [ onaddstream ]
>
>
> onaddstream should be fired upon the receipt of the offer, so that B can
> know what kind of stream to add to the PeerConnection.

I see problems with creating an incoming MediaStream at that early stage 
since stuff can have impact on the negotiation after that point.

There are also a lot of other ways to get the same information. The 
A-side can tell the B-side how it wants to communicate during the find 
and connect phase. It's also possible to read directly from the SDP 
offer in an advanced case.

>     // send answer to A
>
>     A:
>     // process incoming answer
>     processSignalingMessage("SDP ANSWER <id>\n<sdp>")   // id is not local
>
>     // media flowing from A to B
>
>     [ signalingCallback ] "SDP ACK <id>\n<empty sdp>"
>
>
>
>     // A has all info needed to receive
>     [ onaddstream ]
>
>     // send ack to B
>
>     B:
>     // process incoming ack
>     processSignalingMessage("SDP ACK <id>\n<empty sdp>")   // id is not
>     local
>
>     // media flowing from B to A
>
>
> I think media needs to start flowing once B sends its answer to A, or
> B's initial media may be lost.

In that case, A will receive media without knowing which MediaStream to 
map the media to if there are more than one.

/Adam

Received on Monday, 13 February 2012 16:31:49 UTC