JSEP1 BRANCH - WebRTC 1.0: Real-time Communication Between Browsers

3.2.1 MediaStream

The MediaStream() constructor takes two arguments. The arguments are two lists with MediaStreamTrack objects which will be used to construct the audio and video track lists of the new MediaStream object. When the constructor is invoked, the UA must run the following steps:

1. Let audioTracks be the constructor's first argument.

2. Let videoTracks be the constructor's second argument.

3. Let stream be a newly constructed MediaStream object.

4. Set stream's label attribute to a newly generated value.

5. If audioTracks is not null, then run the following sub steps for each element track in audioTracks:

   1. If track is of any other kind than "audio", then throw a SyntaxError exception.

   2. If track has the same underlying source as another element in stream's audio track list, then abort these steps.

   3. Add track to stream's audio track list.

6. If videoTracks is not null, then run the following sub steps for each element track in videoTracks:

   1. If track is of any other kind than "video", then throw a SyntaxError exception.

   2. If track has the same underlying source as another element in stream's video track list, then abort these steps.

   3. Add track to stream's video track list.

A MediaStream can have multiple audio and video sources (e.g. because the user has multiple microphones, or because the real source of the stream is a media resource with many media tracks). The stream represented by a MediaStream thus has zero or more tracks.

The tracks of a MediaStream are stored in two track lists represented by MediaStreamTrackList objects; one for audio tracks and one for video tracks. The two track lists must contain the MediaStreamTrack objects that correspond to the tracks of the stream. The relative order of all tracks in a user agent must be stable. Tracks that come from a media resource whose format defines an order must be in the order defined by the format; tracks that come from a media resource whose format does not define an order must be in the relative order in which the tracks are declared in that media resource. Within these constraints, the order is user-agent defined.

An object that reads data from the output of a MediaStream is referred to as a MediaStream consumer. The list of MediaStream consumers currently include the media elements, PeerConnection and MediaStreamRecorder .

MediaStream consumers be able to handle tracks being added and removed. This behavior is specifier per consumer.

A new media component may be associated with an existing MediaStream . This happens, e.g., on the A-side when the B-side adds a new MediaStreamTrack object to one of the track lists of a MediaStream that is being sent over a PeerConnection . If this happens for the reason exemplified, or for any other reason than the add() method being invoked locally on a MediaStreamTrackList or tracks are being added as the stream is created (i.e. the stream is initialized with tracks), the user agent must run the following steps:

1. Create a MediaStreamTrack object track to represent the new media component.

2. If track's kind attribute equals "audio", add it to the MediaStream object's audioTracks MediaStreamTrackList object.

3. If track's kind attribute equals "video", add it to the MediaStream object's videoTracks MediaStreamTrackList object.

4. Fire a track event named addtrack with the newly created track at the MediaStreamTrackList object.

An existing media component may also be disassociated from a MediaStream . If this happens for any other reason than the remove() method being invoked locally on a MediaStreamTrackList or the stream is being destroyed, the user agent must run the following steps:

1. Let track be the MediaStreamTrack object representing the media component about to be removed.

2. Remove track from the MediaStreamTrackList object.

3. Fire a track event named removetrack with track at the MediaStreamTrackList object.

A MediaStream object is said to be finished when all tracks belonging to the stream have ended. When this happens for any reason other than the stop() method being invoked, the user agent must queue a task that runs the following steps:

1. If the object's ended attribute has the value true already, then abort these steps. (The stop() method was probably called just before the stream stopped for other reasons, e.g. the user clicked an in-page stop button and then the user agent provided stop button.)

2. Set the object's ended attribute to true.

3. Fire a simple event named ended at the object.

If the end of the stream was reached due to a user request, the task source for this task is the user interaction task source. Otherwise the task source for this task is the networking task source.

[Constructor (MediaStreamTrackList? audioTracks, MediaStreamTrackList? videoTracks)]
interface MediaStream {
    readonly attribute DOMString            label;
    readonly attribute MediaStreamTrackList audioTracks;
    readonly attribute MediaStreamTrackList videoTracks;
    MediaStreamRecorder record ();
             attribute boolean              ended;
             attribute Function?            onended;
};

MediaStream implements EventTarget;

3.2.2 LocalMediaStream

Before the web application can access the users media input devices it must let getUserMedia() [GETUSERMEDIA] create a LocalMediaStream . Once the application is done using, e.g., a webcam and a microphone, it may revoke its own access by calling stop() on the LocalMediaStream .

A web application may, once it has access to a LocalMediaStream , use the MediaStream() constructor to construct additional MediaStream objects. Since a derived MediaStream object is created from the tracks of an existing stream, it cannot use any media input devices that have not been approved by the user.

interface LocalMediaStream : MediaStream {
    void stop ();
};

3.2.3 MediaStreamTrack

A MediaStreamTrack object represents a media source in the user agent. Several MediaStreamTrack objects can represent the same media source, e.g., when the user chooses the same camera in the UI shown by two consecutive calls to getUserMedia() [GETUSERMEDIA].

A MediaStreamTrack object can reference its media source in two ways, either with a strong or a weak reference, depending on how the track was created. For example, a track in a MediaStream , derived from a LocalMediaStream with the MediaStream() constructor, has a weak reference to a local media source, while a track in a LocalMediaStream has a strong reference. This means that a track in a MediaStream , derived from a LocalMediaStream , will end if there is no non-ended track in a LocalMediaStream which references the same local media source. A reference to a non-local media source as, e.g., an RTP source, is always strong.

The concept with strong and weak references to media sources allows the web application to derive new MediaStream objects from LocalMediaStream objects (created via getUserMedia() [GETUSERMEDIA]), and still be able to revoke all given permissions with LocalMediaStream.stop() .

A MediaStreamTrack object is said to end when the user agent learns that no more data will ever be forthcoming for this track.

When a MediaStreamTrack object ends for any reason (e.g. because the user rescinds the permission for the page to use the local camera, or because the data comes from a finite file and the file's end has been reached and the user has not requested that it be looped, or because the track belongs to a MediaStream that comes from a remote peer and the remote peer has permanently stopped sending data, or because the UA has instructed the track to end for any reason, or because the reference count of the track's underlying media source has reached zero, it is said to be ended. When track instance track ends for any reason other than stop() method being invoked on the LocalMediaStream object that represents track, the user agent must queue a task that runs the following steps:

1. If the track's readyState attribute has the value ENDED (2) already, then abort these steps.

2. Set track's readyState attribute to ENDED (2).

3. Fire a simple event named ended at the object.

interface MediaStreamTrack {
    readonly attribute DOMString      kind;
    readonly attribute DOMString      label;
             attribute boolean        enabled;
    const unsigned short LIVE = 0;
    const unsigned short MUTED = 1;
    const unsigned short ENDED = 2;
    readonly attribute unsigned short readyState;
             attribute Function?      onmute;
             attribute Function?      onunmute;
             attribute Function?      onended;
};

3.2.4 AudioMediaStreamTrack

The AudioMediaStreamTrack is a specialization of a normal MediaStreamTrack that only carries audio and is extended to have the capability to send and/or receive DTMF codes.

interface AudioMediaStreamTrack : MediaStreamTrack {
    readonly attribute boolean canInsertDTMF;
    void insertDTMF (DOMString tones, optional long duration);
};

3.2.5 MediaStreamTrackList

A MediaStreamTrackList object's corresponding MediaStream refers to the MediaStream object which the current MediaStreamTrackList object is a property of.

interface MediaStreamTrackList {
    readonly attribute unsigned long length;
    MediaStreamTrack item (unsigned long index);
    void             add (MediaStreamTrack track);
    void             remove (MediaStreamTrack track);
             attribute Function?     onaddtrack;
             attribute Function?     onremovetrack;
};

3.2.6 MediaStreamRecorder

The MediaStreamRecorder needs to be able to handle the case that arises when changes are made to the MediaStreamTrackList objects of the MediaStream being recorded; e.g., a new track is added as a result of the add() method being invoked.

interface MediaStreamRecorder {
    voice getRecordedData (BlobCallback? callBack);
};

3.2.7 BlobCallback

[Callback, NoInterfaceObject]
interface BlobCallback {
    void handleEvent (Blob blob);
};

3.2.8 URL

partial interface URL {
    static DOMString createObjectURL (MediaStream stream);
};

4.1.1 Attributes

iceState of type IceState, readonly

The iceState attribute must return the state of the PeerConnection ICE Agent ICE state.

localDescription of type DOMString, readonly

The localDescription method returns a copy of the current local configuration, i.e. the answer that was most recently passed to setLocalDescription, plus any local candidates that have been generated by the ICE Agent since then.

A null object will be returned if the local description has not yet been set with an answer.

localStreams of type array of MediaStream, readonly

Returns a live array containing the local streams (those that were added with addStream() ).

onaddstream of type Function, nullable

This event handler, of event handler event type addstream , must be supported by all objects implementing the PeerConnection interface.

onconnecting of type Function, nullable

This event handler, of event handler event type connecting , must be supported by all objects implementing the PeerConnection interface.

onicechange of type Function, nullable

This event handler, of event handler event type icechange , must be supported by all objects implementing the PeerConnection interface. It is called any time the iceState changes.

onopen of type Function, nullable

This event handler, of event handler event type open , must be supported by all objects implementing the PeerConnection interface.

onremovestream of type Function, nullable

This event handler, of event handler event type removestream , must be supported by all objects implementing the PeerConnection interface.

onstatechange of type Function, nullable

This event handler, of event handler event type open , must be supported by all objects implementing the PeerConnection interface. It is called any time the readyState changes.

readyState of type PeerState, readonly

The readyState attribute must return the PeerConnection object's PeerConnection readiness state.

remoteDescription of type DOMString, readonly

The remoteDescription method returns a copy of the current remote configuration, i.e. the answer that was most recently passed to setRemoteDescription, plus any remote candidates that have been supplied via addIceCandidate since then.

A null object will be returned if the remote description has not yet been set with an answer.

remoteStreams of type array of MediaStream, readonly

Returns a live array containing the streams that the remote streams. (those that were added by the remote side).

This array is updated when addstream and removestream events are fired.

4.1.2 Methods

addIceCandidate

The addIceCandidate method provides a remote candidate to the ICE Agent, which will be added to the remote description. If startIce has been called, connectivity checks will be sent to the new candidates (as long as they would not be restricted by the "media-enum-relay-only" constraint. This call will result in a change to the state of the ICE Agent, and may result in a change to media state if it results in connectivity being established.

The format of the candidate is TODO (same as line with ICE info in SDP)

A TBD exception will be thrown if candidate parameter is malformed.

Parameter	Type	Nullable	Optional	Description
candidate	IceCandidate	✘	✘

Return type: void

addStream

Attempts to starting sending the given stream to the remote peer.

When the other peer starts sending a stream in this manner, an addstream event is fired at the PeerConnection object.

When the addStream() method is invoked, the user agent must run the following steps:

1. If the PeerConnection object's PeerConnection readiness state is CLOSED (3), throw an INVALID_STATE_ERR exception.

2. If stream is already in the PeerConnection object's localStreams object, then abort these steps.

3. Add stream to the end of the PeerConnection object's localStreams object.

4. Return from the method.

5. Parse the constraints provided by the application and apply them to the MediaStream, if possible. NOTE - need to deal with throwing an exception here.

6. Have the PeerConnection add a media stream for stream the next time the user agent provides a stable state. Any other pending stream additions and removals must be processed at the same time.

Parameter	Type	Nullable	Optional	Description
stream	MediaStream	✘	✘
constraints	MediaConstraints	✘	✔

Return type: void

close

When the close() method is invoked, the user agent must run the following steps:

1. If the PeerConnection object's PeerConnection readiness state is CLOSED (3), throw an INVALID_STATE_ERR exception.

2. Destroy the PeerConnection ICE Agent, abruptly ending any active ICE processing and any active streaming, and releasing any relevant resources (e.g. TURN permissions).

3. Set the object's PeerConnection readiness state to CLOSED (3).

The localStreams and remoteStreams objects remain in the state they were in when the object was closed.

No parameters.

Return type: void

createAnswer

The createAnswer method generates a [SDP] answer with the supported configuration for the session that is compatible with the parameters supplied in offer. Like createOffer, the returned blob contains descriptions of the local MediaStreams attached to this PeerConnection, the codec/RTP/RTCP options negotiated for this session, and any candidates that have been gathered by the ICE Agent. The constraints parameter may be supplied to provide additional control over the generated answer.

As an answer, the generated SDP will contain a specific configuration that specifies how the media plane should be established. For each SDP line, the generation of the SDP must follow the appropriate process for generating an answer. Session descriptions generated by createAnswer must be immediately usable by setLocalDescription without generating an error; like createOffer, the returned description should reflect the current state of the system. Calling this method is not required.

A TBD exception is thrown if the constraints parameter is malformed, or the offer parameter is malformed.

Parameter	Type	Nullable	Optional	Description
offer	DOMString	✘	✘
constraints	MediaConstraints	✘	✔

Return type: DOMString

createOffer

The createOffer method generates a blob of SDP that contains a RFC offer with the supported configurations for the session, including descriptions of the local MediaStreams attached to this PeerConnection, the codec/RTP/RTCP options supported by this implementation, and any candidates that have been gathered by the ICE Agent. The constraints parameter may be supplied to provide additional control over the offer generated.

As an offer, the generated SDP will contain the full set of capabilities supported by the session (as opposed to an answer, which will include only a specific negotiated subset to use); for each SDP line, the generation of the SDP must follow the appropriate process for generating an offer. In the event createOffer is called after the session is established, createOffer will generate an offer that is compatible with the current session, incorporating any changes that have been made to the session since the last complete offer-answer exchange, such as addition or removal of streams. If no changes have been made, the offer will be include the capabilities of the current local description.

Session descriptions generated by createOffer must be immediately usable by setLocalDescription without causing an error; if a system has limited resources (e.g. a finite number of decoders), createOffer should return an offer that reflects the current state of the system, so that setLocalDescription will succeed when it attempts to acquire those resources. Calling this method is not required.

Parameter	Type	Nullable	Optional	Description
constraints	MediaConstraints	✘	✘

Return type: DOMString

removeStream

Stops sending the given stream to the remote peer.

When the other peer stops sending a stream in this manner, a removestream event is fired at the PeerConnection object.

When the removeStream() method is invoked, the user agent must run the following steps:

1. If the PeerConnection object's PeerConnection readiness state is CLOSED (3), throw an INVALID_STATE_ERR exception.

2. If stream is not in the PeerConnection object's localStreams object, then abort these steps.

3. Remove stream from the PeerConnection object's localStreams object.

4. Return from the method.

5. Have the PeerConnection remove the media stream for stream the next time the user agent provides a stable state. Any other pending stream additions and removals must be processed at the same time.

Parameter	Type	Nullable	Optional	Description
stream	MediaStream	✘	✘

Return type: void

setLocalDescription

The setLocalDescription method instructs the PeerConnection to apply the supplied [SDP] blob as the local offer or answer. The type parameter indicates whether the blob should be processed as an offer, provisional answer, or final answer.

This API changes the local media state; among other things, it sets up local resources for receiving and decoding media. In order to successfully handle scenarios where the application wants to offer to change from one media format to a different, incompatible format, the PeerConnection must be able to simultaneously support use of both the old and new local descriptions (e.g. support codecs that exist in both descriptions) until a final answer is received, at which point the PeerConnection can fully adopt the new local description, or roll back to the old description if the remote side denied the change.

Changes to the state of media transmission will occur when a final answer is successfully applied.

A TBD exception is thrown if sdp is invalid. A TBD exception is thrown if there are insufficient local resources to apply the sdp.

Parameter	Type	Nullable	Optional	Description
action	SdpType	✘	✘
sdp	DOMString	✘	✘

Return type: void

setRemoteDescription

The setRemoteDescription method instructs the PeerConnection to apply the supplied [SDP]. As in setLocalDescription, the action parameter indicates how the blob should be processed. This API changes the local media state; among other things, it sets up local resources for sending and encoding media.

Changes to the state of media transmission will occur when a final answer is successfully applied.

A TBD exception is thrown if the sdp parameter is invalid. A TBD exception is thrown if there are insufficient local resources to apply the SDP.

Parameter	Type	Nullable	Optional	Description
action	SdpType	✘	✘
sdp	DOMString	✘	✘

Return type: void

startIce

The startIce method starts or updates the ICE Agent process of gathering local candidates and pinging remote candidates. If there is a mandatory constraint called "media-enum-relay-only" and it is set to true, the ICE engine must only use candidates that are thought a relay servers such as a TURN server. This can be used to limit the use to TURN candidates by a callee to avoid leaking location information prior to the call being accepted.

This call may result in a change to the state of the ICE Agent, and may result in a change to media state if it results in connectivity being established.

A TBD exception will be thrown if constraints is malformed.

Parameter	Type	Nullable	Optional	Description
constraints	MediaConstraints	✘	✔

Return type: void

4.2.1 Methods

handleEvent

Def TBD

Parameter	Type	Nullable	Optional	Description
message	DOMString	✘	✘
source	PeerConnection	✘	✘

Return type: void

7.1.1 Attributes

track of type MediaStreamTrack, readonly, nullable

The track attribute represents the MediaStreamTrack object associated with the event.

7.1.2 Dictionary MediaStreamTrackEventInit Members

track of type MediaStreamTrack, nullable

-

7.2.1 Attributes

stream of type MediaStream, readonly, nullable

The stream attribute represents the MediaStream object associated with the event.

7.2.2 Dictionary MediaStreamEventInit Members

stream of type MediaStream, nullable

-