The LocalRealtimeMediaStream Object

The LocalRealtimeMediaStream represents a locally-sourced stream of real-time media.

The constructor for LocalRealtimeMediaStream ignores any information in the RealtimeMediaDescription that it is passed that is not applicable to the provided track.

The RemoteRealtimeMediaStream Object

A RemoteRealtimeMediaStream represents a stream of real-time media that originates on the network.

The constructor to RemoteRealtimeMediaStream constructs a MediaStreamTrack based on the provided RealtimeMediaDescription. If the browser is unable to construct this object, then an error is thrown. This error might be thrown if the media description includes information for both audio and video tracks.

attribute UnknownPacketTypeEventHandler? onunknownpackettype

Called when an unknown packet type arrives on the stream.

The RealtimeMediaDescriptionDictionary Object

Since a media description is likely to be exchanged over a signaling channel, this object represents the value of a RealtimeMediaDescription without the corresponding methods.

sequence<RtpStreamDescription> streams

A description of the RTP streams that comprise the real-time stream.

DOMString cname

The RTCP Canonical Name (CNAME) used to identify this browser as a source of media in RTCP.

sequence<RtpExtension>? rtpExceptions = null

The set of RTP extensions that are used by the described stream(s). Omission of the attribute indicates that extensions are not supported, an empty set indicates that extensions are supported but ignored.

sequence<RtcpFeature> rtcpFeatures

The set of RTCP features that are used by the described stream(s).

sequence<CodecDescription> codecs

A description of the set of codecs that are used by the described stream(s).

byte priority = 3

The relative priority of the stream.

The RtpStreamDescription Object

A RtpStreamDescription describes a single codec and how it is applied to both encoding or decoding. This includes the packet type for the codec and any codec-specific parameters.

unsigned int ssrc

Each RTP stream has a unique, 32-bit synchronization source (SSRC). This value is used to identify a single stream in an RTP session. The browser allocates a new, random SSRC from the set of available SSRC values for each stream, though this value can be overridden by the application. The browser MAY alter this value if it detects an SSRC collision within the RTP session.

unsigned long bandwidth?

The maximum bandwidth, in bits per second, allowed for this stream. This instructs the browser to constrain the stream to fit within this limit. The browser is able to choose the method it uses to meet this constraint.

The RtpExtension Object

RTP Extensions [[!RFC5285]] enables the addition of extension information to the RTP header. This enumeration identifies specific extensions that might be enabled. A set of these values may be used to describe each real-time stream. Omitting the set indicates that header extensions cannot be used on the stream.

enum RtpExtension {
    "rapid-sync", "c2m-alevel", "m2c-alevel"
};
        

rapid-sync

The RTP extension that enables rapid synchronization [[!RFC6051]].

c2m-alevel

The client-to-mixer audio level extension [[!RFC6464]] with mandatory header encryption.

m2c-alevel

The mixer-to-client audio level extension [[!RFC6465]] with mandatory header encryption.

The RtcpFeature Object

The real-time feedback channel provided by RTCP [[!RFC3550]] supports a number of optional features. This enumeration identifies specific features. A set of these values are used to describe each real-time stream.

enum RtcpFeature {
    "avpf", "nack", "pli", "sli", "rpsi", "reduced", "rtx", "fir", "tmmbr", "tstr"
};
          

avpf

The extended profile for RTCP-based feedback, AVPF [[!RFC4585]].

nack

The RTCP generic NACK [[!RFC4585]] feedback feature.

pli

RTCP picture loss indication (PLI) message [[!RFC4585]].

sli

RTCP slice loss indication (SLI) message [[!RFC4585]].

rpsi

RTCP reference picture selection indication (RPSI) message [[!RFC4585]].

reduced

Reduced sized RTCP [[!RFC5506]].

rtx

RTP retransmission [[!RFC4588]].

fir

RTCP full intra-frame request (FIR) message [[!RFC5104]].

tmmbr

Temporary maximum media stream bit rate request (TMMBR) [[!RFC5104]].

tstr

Temporal-spatial trade-off (TSTR) [[!RFC5104]].

The CodecDescription Object

A CodecDescription describes a single codec and how it is applied to both encoding or decoding. This includes the packet type for the codec and any codec-specific parameters.

When generating a RealtimeMediaDescription, the browser MUST specify values for all fields. Values can be omitted from RealtimeMediaDescriptionDictionary values, which allows the browser to select values for any unspecified values.

DOMString type

The MIME media type for the codec. For example, video/H264 or audio/PCMA.

byte packetType

The packet type that is used for this particular codec.

unsigned long clockRate

The clock rate for this codec description, specified in Hertz.

unsigned short? ptime

The packet time for this particular codec. Used to specify the packet time where a choice exists.

unsigned short? maxPtime

The maximum packet time for this particular codec. Used for codecs with a variable packet time.

boolean silenceSuppression = true

For audio codecs that support silence suppression, this value determines if the codec is permitted to perform silence suppression.

DOMString fmtp = ""

Format-specific parameters for the codec, taken from the SDP definition for the codec.

The RealtimeMediaDescriptionConstraints Object

The constraints structure is defined in [[!GETUSERMEDIA]]. RealtimeMediaDescriptionConstraints uses this scheme with a different set of possible constraints.

The following constraints are defined:

rtcp

A set of RTCP features (in an array) that contains all the features that are permitted and desired. This constraint can be met if the RTCP features on all streams are exactly the provided set.

silenceSuppression

A boolean value that determines the value for the silenceSuppression parameter of the CodecDescription.

bandwidth

An unsigned long value for the overall bandwidth allocated to the stream (or set of streams).

Real-Time Transport with RealtimeTransportBuilder

The RealtimeTransportBuilder provides a simple interface for the creation of peer-to-peer real-time transport.

Operation of the RealtimeTransportBuilder is straightforward. The constructor accepts the options that include the RealtimeTransportOptions used to construct a RealtimeTransport. After calling start(), the transport builder produces events when ICE candidates (RealtimePorts) are gathered, and when a transport is successfully created. The following example illustrates its use.

var options = { transport: transportOptions, stun: stunServer };
var builder = new RealtimeTransportBuilder(options);
builder.onport = function(e) {
    signaling.send('port', e.port);
};
signaling.onport = function(port) {
    builder.addRemote(port);
};
builder.onconnect = function(e) {
    e.target.stop();
    gotTransport(e.transport);  // at which point streams can be added, etc...
};
builder.start();
        

Real-Time Transport without RealtimeTransportBuilder

Advanced uses of the transport API can establishing real-time media transport by interacting with the following two objects:

• The RealtimePort is the end point for a real-time transport. This represents a UDP port, opened by the browser. The RealtimePort encapsulates the functions necessary to perform an ICE negotiation [[!RFC5245]]. In ICE terms, a RealtimePort is a local candidate.
• The RealtimeTransport is a peer-to-peer bearer for secured media. It binds a local RealtimePort to a remote RealtimePort and enables the bidirectional exchange of media packets.

In order to establish a transport between a local peer and remote peer, the following process is applied:

1. The local peer opens one or more real-time ports.
2. The local peer then has to learn of the ports that its remote peer has opened. This uses a signaling channel specific to the application. For instance, a web application could use previously HTTP requests or Websockets connections for this purpose.
3. A testing process is used to find a local and remote port pair (a candidate pair) that can exchange UDP packets. One or more connectivity checks are made from different local ports toward different remote ports. A successful connectivity check indicates that packets can reach the peer and that the peer consents to receive packets.
4. Finally, a real-time transport is established on the pair of ports. A security context is established so that secured media packets are able to flow in both directions between peers. RealtimeMediaStreams can then be added to the transport.

In the first step of this scenario one or more RealtimePort objects are created in different ways. The browser allocates a local UDP port on each of the available network interfaces and returns these to the application.

    RealtimePort.openLocalPorts(function(ports) {
        signalingChannel.send('candidates', ports);
        localCandidates.push(ports);
    });
        

Each real-time port consists of an IP address, a port number, a username fragment, a password and a priority. This information is exchanged with the remote peer using whatever signaling mechanism is chosen by the application.

Other forms of real-time port might be necessary for successful traversal of middleboxes like firewalls and NATs. For instance, a relay port (i.e., relay candidate) uses a relay server to forward packets. Because the relay server is publically reachable, many of the concerns with middlebox traversal do not apply to it, enabling communication in more challenging network conditions.

    RealtimePort.allocateRelay(relay, function(error, relayCandidates) {
        if (!error) {
            signalingChannel.send('candidate', relayCandidate);
        }
    });
        

Local and remote real-time ports are paired and connectivity checking is used for each pair to determine which ports can be used for exchanging packets.

    function receivePeerCandidates(remotePorts) {
        var candidatePairs = localCandidates.forEach(function(local) {
            local.addEventListener('checksuccess', checkSuccessCallback);
            return remotePorts.forEach(function(remote) {
                local.check(remote);
            });
        });
    }
        

Of the successful connectivity checks, one or more pairs are selected. A RealtimeTransport is then created from the pair of ports.

    function checkSuccessCallback(event) {
        var local = event.target;
        var remote = event.remote;
        RealtimeTransport.createTransport(local, remote, options, function(err, transport) {
            realtimeTransport = transport;
        });
    }
        

The resulting transport object can be used to securely exchange media with the peer.

The RealtimeTransportBuilder Object

A RealtimeTransportBuilder is responsible for constructing a peer-to-peer transport using ICE.

readonly attribute sequence<RealtimePort> local

The set of gathered RealtimePorts in priority order (highest priority first). These ports can be interacted with directly.

readonly attribute sequence<RemoteRealtimePort> remote

The set of RemoteRealtimePorts provided to the addRemote() method, sorted in priority order.

readonly attribute RealtimeTransport? transport

The current active transport, if one has been successfully created. This value will be null prior to the successful completion of processing.

void start()

Starts or restarts the transport builder. The transport builder will perform the ICE process as though for a single component.

It is possible invoke this method multiple times, where each subsequent invocation triggers a restart of ICE processing. While running, the builder will automatically restart the process if it detects network events that necessitate a restart.

void stop()

Stops the transport builder. Any ports or transports that may have been opened while the builder was running remain open and can continue to be used.

void closeAll()

Stops the builder and closes all ports and transports that may have been created.

[TreatNonCallableAsNull] attribute RealtimeTransportConnectEventHandler? onconnect

The connect event is fired when a transport is successfully created. This event could fire multiple times during the operation of the transport builder if a transport with a higher priority is selected, or network changes cause the current transport to fail.

[TreatNonCallableAsNull] attribute RealtimePortEventHandler? onport

The port event is fired when a local port is gathered.

attribute EventHandler onerror

The error is fired by a transport builder when no candidates can be gathered or when transport creation fails.

callback RealtimeTransportConnectEventHandler = void (RealtimeTransportConnectEvent event);
          

callback RealtimePortEventHandler = void (RealtimePortEvent event);
          

DerivedRealtimePort implements RealtimePort;
          

The RealtimeTransport Object

static void createTransport (RealtimePort local, RemoteRealtimePort remote, TransportOptions options, TransportCallback callback)

Factory method for creating a transport.

readonly attribute RealtimePort localPort

The local real-time port that is used for this transport.

readonly attribute RemoteRealtimePort remotePort

The remote real-time port that is used for this transport.

readonly attribute boolean dtls

Whether key negotiation for the transport is being controlled using DTLS.

readonly attribute RemoteCertificateInformation? remoteCertificate

Information about the certificate offered by the remote peer. This attribute is only set if the transport is using DTLS for key negotiation.

readonly attribute SrtpSecurityDescription? outboundSdes

The SRTP security description for outbound media.

readonly attribute SrtpSecurityDescription? inboundSdes

The SRTP security description for inbound media.

readonly attribute boolean open

Whether the transport is currently open and able to receive data. Newly created transports always start with this value set to true. Note that absence or expiry of consent from the remote peer can cause a transport to be unable to send on an open transport.

readonly attribute unsigned long bandwidth

The maximum available bandwidth on this transport, as detected by the browser. This value can change over time. Users of the transport operate within this envelope; packets that exceed this limit within any interval (browser-selected) are discarded.

void close()

This closes the transport.

attribute EventHandler onclose

The close event is fired when the transport is closed.

readonly attribute boolean consent

Whether the transport is currently open and able to send data. This depends on there being a successful consent check toward the remote peer within a browser-configured period. This valid is false when consent toward the remote peer has expired.

attribute EventHandler onconsent

This consent event is fired when consent checks toward the remote peer succeed after having failed.

attribute EventHandler onconsentexpired

The consentexpired event is fired when consent checks toward the remote peer time out and no packets are able to be sent on this transport.

attribute EventHandler onbandwidthchange

The bandwidthchange event is fired when the browser learns of a change in available bandwidth on the transport. The browser MAY limit the rate of notifications, or suppress notification for small (10% or less) changes in available bandwidth.

attribute unsigned long? inactivityInterval

This attribute can be set to determine when the oninactive event is fired.

attribute EventHandler oninactive

The inactive event is fired when the transport has been inactive for inactivityInterval milliseconds. That is, the time since the last packet was received from the peer exceeds the specified time.

[TreatNonCallableAsNull] attribute UnknownSsrcEventHandler? onunknownssrc

The unknownssrc event is fired when an RTP packet arrives on the transport without a corresponding RealtimeMediaStream. Applications that do not signal SSRCs can use this event to recognize a new or changed SSRC and trigger the addition or modification of streams.

The browser MUST buffer packets that arrive with the SSRC until all event handlers have returned or until a RealtimeMediaStream is created to accept the incoming data. This allows an application to ensure that packets are not lost due to SSRC renumbering or absence of SSRC in signaling.

Transport Creation Options

Specifying RealtimeTransportOptions can be used to alter the characteristics of a transport.

RealtimeTransportMode mode = "dtls-srtp"

The mode used for the transport.

SrtpSecurityDescription outboundSdes

A description of the security parameters for outbound streams of RTP media secured with SRTP [[!RFC3711]]. Used only when SRTP keys are negotiated through signaling.

SrtpSecurityDescription inboundSdes

A description of the security parameters for inbound streams of RTP media secured with SRTP [[!RFC3711]]. Used only when SRTP keys are negotiated through signaling.

DtlsRole? role

The role that the local browser takes in the DTLS [[!RFC5764]] handshake, either 'client' or 'server'.

CertificateValidationCallback validate

This callback is triggered when the certificate for the remote peer is received. This allows the application to accept or reject a peer based on the proffered certificate. This is only valid for a DTLS transport. When omitted, the certificates is automatically accepted.

sequence<DOMString> cipherSuites

A set of DTLS cipher suites that can be negotiated for DTLS, using the Description from the TLS cipher suite registry. The browser cannot negotiate any suite that is not included in this set.

enum RealtimeTransportMode {
    "srtp", "dtls-srtp"
};
          

enum DtlsRole {
    "client", "server"
};
          

callback CertificateValidationCallback = boolean (RemoteCertificateInformation remoteCertificate);
          

function validateCertificate(certInfo) {
    return certInfo.isValidForDomain('example.com');
}
          

Transport Creation Callback

This callback function is invoked when a transport is created.

callback TransportCallback = void (DOMString error, RealtimeTransport transport);
        

The callback is called when a RealtimeTransport is created.

If the transport creation was successful, the error argument is null and the transport argument contains the newly created RealtimeTransport.

If the creation was unsuccessful, the error argument contains a string describing the error and the transport argument is null.

Unknown SSRC Event Handler

This event handler is invoked when an SSRC for which there is no corresponding RealtimeMediaStream is observed.

callback UnknownSsrcEventHandler = void (UnknownSsrcEvent event);
        

Unknown SSRC Event

This event indicates that an SSRC arrived for which there was no corresponding RTP stream.

attribute unsigned long ssrc

The SSRC of the unrecognized packet.

attribute unsigned long ssrc

The SSRC of the unrecognized packet.

The RemoteRealtimePort Object

The RemoteRealtimePort object is used to describe a remote real-time port. It is a simple dictionary of values, sufficient to describe how to connect to a remote peer.

DOMString ip

The IP address (v4 or v6) for the real-time port.

[EnforceRange] unsigned short port

The UDP port number for the real-time port.

DOMString ufrag

The ICE username fragment for the port.

DOMString username

The username is used to specify the STUN USERNAME attribute explicitly. If present, this parameter is used in preference to the username fragment. See server reflexive addresses for an explanation of this parameter.

DOMString pwd

The ICE password for the candidate. This should be set unless this is being used to describe a STUN or TURN server that doesn't require authentication.

unsigned long priority = 0

The ICE priority for the candidate.

The RealtimePort Object

A local RealtimePort provides limited access to the browsers STUN [[!RFC5389]] implementation which provides a means to trigger connectivity checking, the gathering of server reflexive addresses and allocation of TURN relays.

The application cannot select or alter the values chosen for local ports. The browser MAY select any UDP port number, and MUST select values for ufrag and pwd that have sufficient random entropy that they cannot be easily guessed. This guarantees that only entities that receive this information are able to successfully make a connectivity check. It also allows peers to identify unique browsers.

readonly attribute DOMString ip

The IP address (v4 or v6) for the real-time port.

readonly attribute unsigned short port

The UDP port number for the real-time port.

readonly attribute DOMString ufrag

The ICE username fragment for the port.

readonly attribute DOMString pwd

The ICE password for the port.

readonly attribute unsigned long priority

The ICE priority for the candidate. This priority allows for prioritization of checks. This parameter is added to all checks sent by the candidate.

static void openLocalPorts (RealtimePortCallback callback)

Open ports on the local host. One RealtimePort is opened for each network interface on the host. Calls the provided callback when complete, passing the ports that were opened as a result of the function call. Ports are provided to the callback in priority order.

static void allocateRelay (TurnServer turnServer, RealtimePortCallback callback)

Allocate a relay port on a TURN server using a newly created local base.

void allocateRelay (TurnServer turnServer, RelayCallback callback)

Allocate a relay port on a TURN server over UDP using this local real-time port as the base.

[TreatNonCallableAsNull] attribute EventHandler onnetworkchange

An event handler for being informed of changes to network connectivity. Note: This is a candidate for moving to navigator or even the network information API.

[TreatNonCallableAsNull] attribute ConnectivityCheckEventHandler? onchecksent

Called when a connectivity check is sent to a remote peer. The event passed MUST NOT include the STUN Binding request.

[TreatNonCallableAsNull] attribute ConnectivityCheckEventHandler? onchecksuccess

Called when a valid connectivity check response is received from a remote peer.

[TreatNonCallableAsNull] attribute ConnectivityCheckEventHandler? onremotecheck

Called when a remote peer successfully performs a connectivity check against this port.

void setServerAttributes(sequence serverAttributes)

Define a set of attributes that the port will add to STUN Binding check responses. Attributes with a value will be sent verbatim, attributes with no value set will be echoed from the request, but only if the request includes that attribute.

readonly attribute sequence serverAttributes

The set of attributes that the port will add to STUN Binding check responses.

[TreatNonCallableAsNull] attribute EventHandler onclose

Invoked when the real-time port is closed.

readonly attribute boolean open

Whether the port is open and can be used. This attribute is true when the object is passed to the application and false after the close method is invoked.

long check (RemoteRealtimePort remote, optional StunAttribute... attributes)

Perform a connectivity check on the remote port. Returns a handle on the check that can be used to cancel it.

void cancelCheck (long checkHandle)

Cancel an outstanding or enqueued connectivity check. This guarantees that a onchecksuccess event is never triggered for the specified check.

boolean status (RemoteRealtimePort remoteRealtimePort)

Query the status of a particular remote port with respect to this port. Returns true iff the remote port was successfully checked recently. The specific time is based on the browser's configured consent expiration timer.

void close ()

Closes the port and any associated open transports that use the port.

callback RealtimePortCallback = void (DOMString? error, sequence<RealtimePort> ports);
          

callback ConnectivityCheckEventHandler = void (ConnectivityCheckEvent event);
          

RealtimePort Lifecycle

A local RealtimePort is created by the browser in response to a call to openLocalPorts(callback) or allocateRelay(callback). Each RealtimePort is given to the application with the open attribute set to true.

    function createRelayCandidate() {
        var relay = { ip: '192.0.2.75', port: 7754, username: 'user', password: 'password1', turn: 'udp' };
        RealtimePort.allocateRelay(relay, function(error, relayCandidates) {
            if (error) {
                reportError(error);
            } else {
                signalingChannel.send('candidate', relayCandidate);
                scheduleConnectivityChecks(relayCandidate);
            }
        });
    }
          

enum TurnRelayTransport {
    "udp", "tcp", "tls"
};
          

Connectivity Checking

To establish a bidirectional transport flow between peers, both are required to successfully complete one or more connectivity check requests. Connectivity checks are used to open ports on NAT and firewall middleboxes, to validate end-to-end connectivity and to establish that the peer is willing to receive packets.

Network Interface Changes

Network interfaces are imperfect and impermanent. They can change, appear and disappear with no warning, especially on mobile devices. Reacting to changes in connectivity is a crucial part of providing good real-time transport.

The networkchange event is registered against a local RealtimePort.

Ideally, this would be globally registered, but since events can't originate on static objects, this is difficult. Again, ideally this is something for the network information API to solve.

The browser fires a networkchange event when it detects a potential change in the status of the network. Events that trigger this callback include changes to open network interfaces and the addition or removal of network interfaces.

Privacy Considerations

Applications that establish peer-to-peer transports require that the IP addresses of a peer are signaled to the remote peer. This can pose a privacy exposure even though an IP address can only be loosely correlated with a person. For instance, it is possible to use IP addresses to determine the physical location of a person.

In some applications, establishing a peer-to-peer transport occurs prior to establishing user consent for the session. This can be necessary to remove the delays associated with transport setup that might otherwise occur after session acceptance. Exposing IP address information prior to acceptance provides the initiator of the session a way to collect the IP address of even an unwilling peer.

Applications are encouraged to only signal relay ports prior to gaining explicit consent from users.