Re: Low Level Javascript API Proposal

On 10/05/2011 04:27 AM, Matthew Kaufman wrote:
> On 9/28/2011 9:46 AM, Neil Stratford wrote:
>>
>> That would certainly be the intention. The idea is really to split 
>> codec and ICE negotiation apart, but retain the ability to combine in 
>> javascript to provide a PeerConnection style SDP based API if required.
>>
>
> This part is key, and I've brought it up in the past. The 
> "PeerConnection" has a legitimate need to participate in the ICE 
> negotiation, and so if ICE candidates are encoded as SDP rather than 
> JSON it makes sense for it to generate and consume that SDP... but it 
> is NOT appropriate for the PeerConnection to also be generating and 
> consuming an SDP offer-answer for codec negotation. The codecs should 
> be different objects than the PeerConnection, usable for other things 
> (like local processing, recording, playback from file, etc.) and so it 
> just doesn't make sense for the PeerConnection to be changing their 
> settings at any time. (Or worse, for the PeerConnection's SDP 
> processing to be the *only* way of changing their settings.)
The relationship between a codec and a PeerConnection will unfortunately 
have to be rather intimate in several ways, I think.

In particular, congestion control will have to have the ability to telll 
a codec to send less data, or the ability to tell a codec that it can 
send more data if it wants to. It may be possible to allow that feedback 
loop to be mediated via the Javascript layer, but I'm not enough up to 
speed on that issue to be able to say for sure yet.

The other thing that is hard to deal with in the proposed low level API 
is inter-stream relationships. If we follow the modeling path so far 
suggested of putting several MediaStream objects' video tracks down one 
RTP session and their audio tracks down another RTP session, the 
relationships between RTP sessions, SSRCs and so on can get complex 
fast, and may uncover several new ways of letting people write apps that 
fail in interesting ways.

Again, I may be worrying too much. Let's talk.

                   Harald

Received on Wednesday, 5 October 2011 12:00:41 UTC