summary of issues raised at W3C meeting

During the one and a half days of WEBRTC meeting and half day of Media Capture Task Force meeting at this week's W3C TPAC, a large number of issues came up, some of which were resolved. These fall into several categories, such as issues for the W3C WG, for other W3C WGs, for the IETF RTCWEB WG, other IETF WGs, and in some cases span more than one. Rather than attempt to categorize each of these items, I will simply list them below for 1) others present to expand upon where I missed or mis-stated something and 2) others to categorize appropriate and/or turn into action items (drafts, etc.)

-- begin list --

Collections of streams within an RTCPeerConnection are currently an array. This leads to the risk that someone will remember the array index, which would prevent removing streams. Solution is to replace this with a different type of collection and accessor. (Likewise, outside of WEBRTC, there needs to be a corresponding change for arrays of tracks within streams)

 Both streams and tracks need both "id" and "label" fields. "id" is guaranteed to be unique and will be used as the "msid" in the SDP. "label" is human-readable.

 AGC and AEC settings will have local effect only, set using constraints.

 There is no API to reject an incoming track (and thus reject from the offer/answer exchange without modifying SDP directly)... even if there were at the track level, this wouldn't map directly to the m= sections of the SDP.

It was suggested that content labels to be added a parameter on a track. Default of none. If specific content type is set then label is set in a new m= section with a=content: set to match the label.

 SDES or not still needs to be decided at IETF. Also things like MTI cipher suites for both SDES and DTLS-SRTP.

 There is apparently a desire to get certain kinds of constraints to the far end of an RTCPeerConnection... treating it as a pipe. So that if a constraint of (say) resolution is placed on a video track that is playing out that constraint would be sent somehow (SDP? RTCP? Application?) to the other end and affect the sending track and ultimately the source device.

 There is a desire to do trickle ICE. This requires a way to signal the desire to do trickle ICE in the SDP in a way that is backwards-compatible with existing RFC3264 O/A and existing non-trickle ICE implementations, while still generating fully 3264-compatible SDP for the trickle case (or updating 3264 and pointing JSEP at the update).

 There is a need to write down what MUST NOT be changed in the SDP that comes out of createOffer (and createAnswer?) before it is put in to setLocalDescription, and what MUST BE ACCEPTED IF CHANGED. This includes complexities such as reassignment of ports in m= line. The rest is permissive to change (see below).

SDP changes that aren't compatible either because they are in the "what MUST NOT be changed" list or in the (default) permissive list but not supported by this browser must be reported as an error. That error must include an error object that has an sdpLineNumber field.

The ICE state machine in the draft is poorly labeled... need a better drawing.

There is a desire to merge the icegatheringchange event into the onicecandidate event.

Rename iceconnect state to iceconnected.

Still open the be decided in media capture task force is what device enumeration we are allowed to do before and after user consent is received.

Stats - Several alternatives were proposed, general consensus to use a javascript structure with some limited hierarchy vs. flat dictionary w/ string keys encoding structure into the strings. Document the ICE stats as an example to test this.

DTMF - Decided to wrap mediastream in a decorator using peerconnection to do the creation/wrapping (so it learns about the peerconnection), NOT add a dtmf api on the peerconnection.

Open issue - at receiver, do all audio streams that have DTMF in the SDP come out decorated as above or undecorated?

Identity proposal was discussed - General opinion was that a hybrid approach ("# 4 = (1+3)") with peeridentity attribute is appropriate. Open issue is how to distinguish between have and don't have identity cases.

There was a desire to propose new names for onaddstream events and maybe others.

Open issue: how does createOffer decide to offer a data channel - not resolved

 Still need to resolve bundle at the IETF.

Still need to resolve media stream ID at the IETF.

Need to specify what happens if ssrc is not provided or media stream ID is not provided in received SDP. Is this an error?

We need a way to set priority for each track/data channel (3 or 4 levels) *and* have that enforced in the network subsystem of UA *and* optionally diffserv mark *and* get the diffserv markings back via an API (and include in SDP?)

createOffer, createAnswer, setLocalDescription, setRemoteDescription all cause callbacks. There is ambiguity created in the state machine if any of these can be called while any other is in the process of acting. Queuing must be implemented to ensure that execution of any of these is prevented until the callback is delivered from the previous call to any of these.

Rollback APIs must be added for both setLocalDescription(offer) and setRemoteDescription(offer) in order to get back to the stable state from which the same or opposite might be called.

Need to strengthen language to show that createOffer() is mandatory to call because without createOffer() you cannot have some of the SDP that MUST be provided to setLocalDescription()

Desire to change the guarantee of how long createOffer() is guaranteed to have its output accepted by setLocalDescription() from "in callback" to "15 seconds" (or something similar to that)

Discussion of changing the defaults for starting ICE from "all" to "none", resolved to staying with "all" and adding contraints (see below).

Decision to add a new constraint on RTCPeerConnection() to ask it to "preheat" a specified number of ICE candidate... the default constraint is 0. Therefore, this also clarifies than candidate gathering will not start at "new RTCPeerConnection" and instead waits until the constraint is set OR some undefined point in the future which still needs clarification (at createOffer or setLocalDescription, but neither is currently a good answer).

Decision to merge the concept of constraints and settings to only have constraints. Constraints always replace previous constraints. (Some interpretations were to set them to the intersection of request and previous.)

 getUserMedia() currently returns nothing, and has async callbacks on success or failure. This is a problem because the callee side of an RTCPeerConnection must either 1) not connect the camera immediately when a call is signaled, and thus respond with recvonly (and send the information to the caller necessary to have it pre-attached to the correct audio and video tags (because the information required to populate the ssrc and msid and send a provision answer doesn't exist), thus preventing playout at the caller even if RTP packets were to be sent to it), or 2) pre-allocate the camera and/or microphone while waiting for a call and tie up the resource. Proposed solution is to change getUserMedia() to return a MediaStream immediately which is a source of no sound and no video... if/when user consent is received this is delivered as an event on that stream, likewise failure to gain consent is an event on that stream. There was some discussion against the idea of returning the MediaStream synchronously in favor of a callback delivering the "empty" MediaStream which was not resolved.

-- end list --

Matthew Kaufman (and compiled with the assistance of Dan Burnett and Martin Thomson)

Received on Friday, 2 November 2012 18:22:14 UTC