Note
This is a fragment of the original document. The purpose is to get reviews on the "RTP media API" section.

Abstract

This document defines a set of ECMAScript APIs in WebIDL to allow media to be sent to and received from another browser or device implementing the appropriate set of real-time protocols. This specification is being developed in conjunction with a protocol specification developed by the IETF RTCWEB group and an API specification to get access to local media devices developed by the Media Capture Task Force.

Status of This Document

This section describes the status of this document at the time of its publication. Other documents may supersede this document. A list of current W3C publications and the latest revision of this technical report can be found in the W3C technical reports index at http://www.w3.org/TR/.

This document was published by the Web Real-Time Communications Working Group as a Working Group Note. If you wish to make comments regarding this document, please send them to public-webrtc@w3.org (subscribe, archives). All comments are welcome.

Publication as a Working Group Note does not imply endorsement by the W3C Membership. This is a draft document and may be updated, replaced or obsoleted by other documents at any time. It is inappropriate to cite this document as other than work in progress.

This document was produced by a group operating under the 5 February 2004 W3C Patent Policy. W3C maintains a public list of any patent disclosures made in connection with the deliverables of the group; that page also includes instructions for disclosing a patent. An individual who has actual knowledge of a patent which the individual believes contains Essential Claim(s) must disclose the information in accordance with section 6 of the W3C Patent Policy.

This document is governed by the 1 August 2014 W3C Process Document.

Table of Contents

1. Introduction

This section is non-normative.

There are a number of facets to video-conferencing in HTML covered by this specification:

This document defines the APIs used for these features. This specification is being developed in conjunction with a protocol specification developed by the IETF RTCWEB group and an API specification to get access to local media devices [GETUSERMEDIA]developed by the Media Capture Task Force. An overview of the system can be found in [RTCWEB-OVERVIEW] and [RTCWEB-SECURITY].

2. Terminology

The EventHandler interface represents a callback used for event handlers as defined in [HTML5].

The concepts queue a task and fires a simple event are defined in [HTML5].

The terms event, event handlers and event handler event types are defined in [HTML5].

The terms MediaStream, MediaStreamTrack, Constraints, and Consumer are defined in [GETUSERMEDIA].

3. RTP media API

The RTP media API lets a web application send and receive MediaStreamTracks over a peer-to-peer connection. Tracks, when added to a PeerConnection, result in signaling; when this signaling is forwarded to a remote peer, it causes corresponding tracks to be created on the remote side.

The actual encoding and transmission of MediaStreamTracks is managed through objects called RTCRtpSenders. Similarly, the reception and decoding of MediaStreamTracks is managed through objects called RTCRtpReceivers.

A RTCPeerConnection object contains a set of RTCRtpSenders, representing tracks to be sent, and a set of RTCRtpReceivers, representing tracks that are to be received on this RTCPeerConnection object. Both of these sets are initialized to empty sets when the RTCPeerConnection object is created.

3.1 RTCPeerConnection Interface Extensions

The RTP media API extends the RTCPeerConnection interface as described below.

partial interface RTCPeerConnection {
    sequence<RTCRtpSender>   getSenders ();
    sequence<RTCRtpReceiver> getReceivers ();
    RTCRtpSender             addTrack (MediaStreamTrack track, MediaStream... streams);
    void                     removeTrack (RTCRtpSender sender);
                attribute EventHandler onaddtrack;
};

3.1.1 Attributes

onaddtrack of type EventHandler,
Note
Should we call the corresponding event "remotetrack", since we already have an event named "addtrack" in the Media Capture and Streams spec?

This event handler, of event handler event type addtrack, MUST be supported by all objects implementing the RTCPeerConnection interface. It is called any time a MediaStreamTrack is added by the remote peer. This will be fired only as a result of setRemoteDescription. onaddtrack happens as early as possible after the setRemoteDescription. This callback does not wait for a given media track to be accepted or rejected via SDP negotiation.

3.1.2 Methods

addTrack

Adds a new track to the RTCPeerConnection, and indicate that it is contained in the specified MediaStreams.

When the addTrack() method is invoked, the user agent MUST run the following steps:

  1. Let connection be the RTCPeerConnection object on which the MediaStreamTrack, track, is to be added.

  2. If connection's RTCPeerConnection signalingState is closed, throw an InvalidStateError exception and abort these steps.

  3. If an RTCRtpSender for track already exists in connection's set of senders, then abort these steps.

  4. Create a new RTCRtpSender for track, add it to connection's set of senders, and return it to the caller.

    5. Note

    A track could have contents that are inaccessible to the application. This can be due to being marked with a peerIdentity option or anything that would make a track CORS cross-origin. These tracks can be supplied to the addTrack method, and have an RTCRtpSender created for them, but content MUST NOT be transmitted, though tracks marked with peerIdentity can be transmitted if they meet the requirements for sending (see isolated streams and RTCPeerConnection).

    All other tracks that are not accessible to the application MUST NOT be sent to the peer, with silence (audio), black frames (video) or equivalently absent content being sent in place of track content.

    Note that this property can change over time.

  5. If connection's RTCPeerConnection signalingState is stable, then fire a negotiationneeded event at connection.

ParameterTypeNullableOptionalDescription
trackMediaStreamTrack✘✘
streamsMediaStream✘✘
Return type: RTCRtpSender
getReceivers

Returns a sequence of RTCRtpReceiver objects representing the RTP receivers that are currently attached to this RTCPeerConnection object.

The getReceivers() method MUST return a new sequence that represents a snapshot of all the RTCRtpReceiver objects in this RTCPeerConnection object's set of receivers. The conversion from the receivers set to the sequence, to be returned, is user agent defined and the order does not have to stable between calls.

No parameters.
Return type: sequence<RTCRtpReceiver>
getSenders

Returns a sequence of RTCRtpSender objects representing the RTP senders that are currently attached to this RTCPeerConnection object.

The getSenders() method MUST return a new sequence that represents a snapshot of all the RTCRtpSenders objects in this RTCPeerConnection object's set of senders. The conversion from the senders set to the sequence, to be returned, is user agent defined and the order does not have to stable between calls.

No parameters.
Return type: sequence<RTCRtpSender>
removeTrack

Removes sender, and its associated MediaStreamTrack, from the RTCPeerConnection.

When the other peer stops sending a track in this manner, a ended event is fired at the MediaStreamTrack object.

When the removeTrack() method is invoked, the user agent MUST run the following steps:

  1. Let connection be the RTCPeerConnection object on which the RTCRtpSender, sender, is to be removed.

  2. If connection's RTCPeerConnection signalingState is closed, throw an InvalidStateError exception.

  3. If sender is not in connection's set of senders, then abort these steps.

  4. Remove sender from connection's set of senders.

  5. If connection's RTCPeerConnection signalingState is stable, then fire a negotiationneeded event at connection.

ParameterTypeNullableOptionalDescription
senderRTCRtpSender✘✘
Return type: void

3.2 RTCRtpSender Interface

The RTCRtpSender interface allows an application to control how a given MediaStreamTrack is encoded and transmitted to a remote peer. When attributes on an RTCRtpSender are modified, the encoding is either changed appropriately, or a negotiation is triggered to signal the new encoding parameters to the other side.

interface RTCRtpSender {
    readonly    attribute MediaStreamTrack track;
};

3.2.1 Attributes

track of type MediaStreamTrack, readonly

The RTCRtpSender.track attribute is the track that is immutably associated with this RTCRtpSender object.

3.3 RTCRtpReceiver Interface

The RTCRtpReceiver interface allows an application to control the receipt of a MediaStreamTrack. When attributes on an RTCRtpReceiver are modified, a negotiation is triggered to signal the changes regarding what the application wants to receive to the other side.

interface RTCRtpReceiver {
    readonly    attribute MediaStreamTrack track;
};

3.3.1 Attributes

track of type MediaStreamTrack, readonly

The RTCRtpReceiver.track attribute is the track that is immutably associated with this RTCRtpReceiver object.

3.4 RTCAddTrackEvent

Note
Consider renaming RTCAddTrackEvent to RTCRemoteTrackEvent.

The onaddtrack event uses the RTCAddTrackEvent interface.

Firing an RTCAddTrackEvent event named e with an MediaStreamTrack track means that an event with the name e, which does not bubble (except where otherwise stated) and is not cancelable (except where otherwise stated), and which uses the RTCAddTrackEvent interface with the track attribute set to track, MUST be created and dispatched at the given target.

dictionary RTCAddTrackEventInit : EventInit {
    RTCRtpReceiver        receiver;
    MediaStreamTrack      track;
    sequence<MediaStream> streams;
};

[ Constructor (DOMString type, RTCAddTrackEventInit eventInitDict)]
interface RTCAddTrackEvent : Event {
    readonly    attribute RTCRtpReceiver   receiver;
    readonly    attribute MediaStreamTrack track;
    sequence<MediaStream> getStreams ();
};

3.4.1 Constructors

RTCAddTrackEvent
readonly attribute RTCRtpReceiver receiver
ParameterTypeNullableOptionalDescription
typeDOMString✘✘
eventInitDictRTCAddTrackEventInit✘✘

3.4.2 Attributes

receiver of type RTCRtpReceiver, readonly

The receiver attribute represents the RTCRtpReceiver object associated with the event.

track of type MediaStreamTrack, readonly

The track attribute represents the MediaStreamTrack object that is associated with the RTCRtpReceiver identified by receiver.

3.4.3 Methods

getStreams

Returns a sequence of MediaStream objects representing the MediaStreams that this event's track is a part of.

No parameters.
Return type: sequence<MediaStream>

3.4.4 Dictionary RTCAddTrackEventInit Members

receiver of type RTCRtpReceiver

TODO

streams of type sequence<MediaStream>

TODO

track of type MediaStreamTrack

TODO

4. Examples and Call Flows

This section is non-normative.

4.1 Simple Peer-to-peer Example

When two peers decide they are going to set up a connection to each other, they both go through these steps. The STUN/TURN server configuration describes a server they can use to get things like their public IP address or to set up NAT traversal. They also have to send data for the signaling channel to each other using the same out-of-band mechanism they used to establish that they were going to communicate in the first place.

Example 1
var signalingChannel = new SignalingChannel();
var configuration = { "iceServers": [{ "url": "stun:stun.example.org" }] };
var pc;

// call start() to initiate
function start() {
    pc = new RTCPeerConnection(configuration);

    // send any ice candidates to the other peer
    pc.onicecandidate = function (evt) {
        if (evt.candidate)
            signalingChannel.send(JSON.stringify({ "candidate": evt.candidate }));
    };

    // let the "negotiationneeded" event trigger offer generation
    pc.onnegotiationneeded = function () {
        pc.createOffer(localDescCreated, logError);
    }

    // once remote video track arrives, show it in the remote video element
    pc.onaddtrack = function (evt) {
        if (evt.track.kind === "video")
          remoteView.srcObject = evt.getStreams()[0];
    };

    // get a local stream, show it in a self-view and add it to be sent
    navigator.mediaDevices.getUserMedia({ "audio": true, "video": true }, function (stream) {
        selfView.srcObject = stream;
        if (stream.getAudioTracks().length > 0)
            pc.addTrack(stream.getAudioTracks()[0], stream);
        if (stream.getVideoTracks().length > 0)
            pc.addTrack(stream.getVideoTracks()[0], stream);
    }, logError);
}

function localDescCreated(desc) {
    pc.setLocalDescription(desc, function () {
        signalingChannel.send(JSON.stringify({ "sdp": pc.localDescription }));
    }, logError);
}

signalingChannel.onmessage = function (evt) {
    if (!pc)
        start();

    var message = JSON.parse(evt.data);
    if (message.sdp)
        pc.setRemoteDescription(new RTCSessionDescription(message.sdp), function () {
            // if we received an offer, we need to answer
            if (pc.remoteDescription.type == "offer")
                pc.createAnswer(localDescCreated, logError);
        }, logError);
    else
        pc.addIceCandidate(new RTCIceCandidate(message.candidate),
            function () {}, logError);
};

function logError(error) {
    log(error.name + ": " + error.message);
}

A. Acknowledgements

The editors wish to thank the Working Group chairs and Team Contact, Harald Alvestrand, Stefan Håkansson and Dominique Hazaël-Massieux, for their support. Substantial text in this specification was provided by many people including Martin Thomson, Harald Alvestrand, Justin Uberti, and Eric Rescorla.

B. References

B.1 Normative references

[GETUSERMEDIA]
Daniel Burnett; Adam Bergkvist; Cullen Jennings; Anant Narayanan. Media Capture and Streams. 3 September 2013. W3C Working Draft. URL: http://www.w3.org/TR/mediacapture-streams/
[HTML5]
Robin Berjon; Steve Faulkner; Travis Leithead; Erika Doyle Navara; Edward O'Connor; Silvia Pfeiffer. HTML5. 16 September 2014. W3C Proposed Recommendation. URL: http://www.w3.org/TR/html5/

B.2 Informative references

[RTCWEB-OVERVIEW]
H. Alvestrand. Overview: Real Time Protocols for Brower-based Applications. 14 February 2014. Active Internet-Draft. URL: http://datatracker.ietf.org/doc/draft-ietf-rtcweb-overview/
[RTCWEB-SECURITY]
Eric Rescorla. Security Considerations for WebRTC. 22 January 2014. Active Internet-Draft. URL: http://datatracker.ietf.org/doc/draft-ietf-rtcweb-security/