JSEP EASY 1 BRANCH - WebRTC 1.0: Real-time Communication Between Browsers

3.2.1 MediaStream

The MediaStream() constructor takes two arguments. The arguments are two lists with MediaStreamTrack objects which will be used to construct the audio and video track lists of the new MediaStream object. When the constructor is invoked, the UA must run the following steps:

1. Let audioTracks be the constructor’s first argument.

2. Let videoTracks be the constructor’s second argument.

3. Let stream be a newly constructed MediaStream object.

4. Set stream’s label attribute to a newly generated value.

5. If audioTracks is not null, then run the following sub steps for each element track in audioTracks:

   1. If track is of any other kind than "audio", then throw a SyntaxError exception.

   2. If track has the same underlying source as another element in stream’s audio track list, then abort these steps.

   3. Add track to stream’s audio track list.

6. If videoTracks is not null, then run the following sub steps for each element track in videoTracks:

   1. If track is of any other kind than "video", then throw a SyntaxError exception.

   2. If track has the same underlying source as another element in stream’s video track list, then abort these steps.

   3. Add track to stream’s video track list.

A MediaStream can have multiple audio and video sources (e.g. because the user has multiple microphones, or because the real source of the stream is a media resource with many media tracks). The stream represented by a MediaStream thus has zero or more tracks.

The tracks of a MediaStream are stored in two track lists represented by MediaStreamTrackList objects; one for audio tracks and one for video tracks. The two track lists must contain the MediaStreamTrack objects that correspond to the tracks of the stream. The relative order of all tracks in a user agent must be stable. Tracks that come from a media resource whose format defines an order must be in the order defined by the format; tracks that come from a media resource whose format does not define an order must be in the relative order in which the tracks are declared in that media resource. Within these constraints, the order is user-agent defined.

An object that reads data from the output of a MediaStream is referred to as a MediaStream consumer. The list of MediaStream consumers currently include the media elements, PeerConnection and MediaStreamRecorder.

MediaStream consumers be able to handle tracks being added and removed. This behavior is specifier per consumer.

A new media component may be associated with an existing MediaStream. This happens, e.g., on the A-side when the B-side adds a new MediaStreamTrack object to one of the track lists of a MediaStream that is being sent over a PeerConnection. If this happens for the reason exemplified, or for any other reason than the add() method being invoked locally on a MediaStreamTrackList or tracks are being added as the stream is created (i.e. the stream is initialized with tracks), the user agent must run the following steps:

1. Create a MediaStreamTrack object track to represent the new media component.

2. If track’s kind attribute equals "audio", add it to the MediaStream object’s audioTracks MediaStreamTrackList object.

3. If track’s kind attribute equals "video", add it to the MediaStream object’s videoTracks MediaStreamTrackList object.

4. Fire a track event named addtrack with the newly created track at the MediaStreamTrackList object.

An existing media component may also be disassociated from a MediaStream. If this happens for any other reason than the remove() method being invoked locally on a MediaStreamTrackList or the stream is being destroyed, the user agent must run the following steps:

1. Let track be the MediaStreamTrack object representing the media component about to be removed.

2. Remove track from the MediaStreamTrackList object.

3. Fire a track event named removetrack with track at the MediaStreamTrackList object.

A MediaStream object is said to be finished when all tracks belonging to the stream have ended. When this happens for any reason other than the stop() method being invoked, the user agent must queue a task that runs the following steps:

1. If the object’s ended attribute has the value true already, then abort these steps. (The stop() method was probably called just before the stream stopped for other reasons, e.g. the user clicked an in-page stop button and then the user-agent-provided stop button.)

2. Set the object’s ended attribute to true.

3. Fire a simple event named ended at the object.

If the end of the stream was reached due to a user request, the task source for this task is the user interaction task source. Otherwise the task source for this task is the networking task source.

[Constructor (MediaStreamTrackList? audioTracks, MediaStreamTrackList? videoTracks)]
interface MediaStream {
    readonly attribute DOMString            label;
    readonly attribute MediaStreamTrackList audioTracks;
    readonly attribute MediaStreamTrackList videoTracks;
    MediaStreamRecorder record ();
             attribute boolean              ended;
             attribute Function?            onended;
};

MediaStream implements EventTarget;

3.2.2 LocalMediaStream

Before the web application can access the users media input devices it must let getUserMedia() [GETUSERMEDIA] create a LocalMediaStream. Once the application is done using, e.g., a webcam and a microphone, it may revoke its own access by calling stop() on the LocalMediaStream .

A web application may, once it has access to a LocalMediaStream, use the MediaStream() constructor to construct additional MediaStream objects. Since a derived MediaStream object is created from the tracks of an existing stream, it cannot use any media input devices that have not been approved by the user.

interface LocalMediaStream : MediaStream {
    void stop ();
};

3.2.3 MediaStreamTrack

A MediaStreamTrack object represents a media source in the user agent. Several MediaStreamTrack objects can represent the same media source, e.g., when the user chooses the same camera in the UI shown by two consecutive calls to getUserMedia() [GETUSERMEDIA].

A MediaStreamTrack object can reference its media source in two ways, either with a strong or a weak reference, depending on how the track was created. For example, a track in a MediaStream, derived from a LocalMediaStream with the MediaStream() constructor, has a weak reference to a local media source, while a track in a LocalMediaStream has a strong reference. This means that a track in a MediaStream, derived from a LocalMediaStream, will end if there is no non-ended track in a LocalMediaStream which references the same local media source. A reference to a non-local media source as, e.g., an RTP source, is always strong.

The concept with strong and weak references to media sources allows the web application to derive new MediaStream objects from LocalMediaStream objects (created via getUserMedia() [GETUSERMEDIA]), and still be able to revoke all given permissions with LocalMediaStream.stop().

A MediaStreamTrack object is said to end when the user agent learns that no more data will ever be forthcoming for this track.

When a MediaStreamTrack object ends for any reason (e.g. because the user rescinds the permission for the page to use the local camera, or because the data comes from a finite file and the file’s end has been reached and the user has not requested that it be looped, or because the track belongs to a MediaStream that comes from a remote peer and the remote peer has permanently stopped sending data, or because the UA has instructed the track to end for any reason, or because the reference count of the track’s underlying media source has reached zero, it is said to be ended. When track instance track ends for any reason other than stop() method being invoked on the LocalMediaStream object that represents track, the user agent must queue a task that runs the following steps:

1. If the track’s readyState attribute has the value ENDED (2) already, then abort these steps.

2. Set track’s readyState attribute to ENDED (2).

3. Fire a simple event named ended at the object.

interface MediaStreamTrack {
    readonly attribute DOMString      kind;
    readonly attribute DOMString      label;
             attribute boolean        enabled;
    const unsigned short LIVE = 0;
    const unsigned short MUTED = 1;
    const unsigned short ENDED = 2;
    readonly attribute unsigned short readyState;
             attribute Function?      onmute;
             attribute Function?      onunmute;
             attribute Function?      onended;
};

3.2.4 AudioMediaStreamTrack

The AudioMediaStreamTrack is a specialization of of a normal MediaStreamTrack that only carries audio and is extended to have the capability to send and/or receive DTMF codes.

interface AudioMediaStreamTrack : MediaStreamTrack {
    readonly attribute boolean canInsertDTMF;
    void insertDTMF (DOMString tones, optional long duration);
};

3.2.5 MediaStreamTrackList

A MediaStreamTrackList object’s corresponding MediaStream refers to the MediaStream object which the current MediaStreamTrackList object is a property of.

interface MediaStreamTrackList {
    readonly attribute unsigned long length;
    MediaStreamTrack item (unsigned long index);
    void             add (MediaStreamTrack track);
    void             remove (MediaStreamTrack track);
             attribute Function?     onaddtrack;
             attribute Function?     onremovetrack;
};

3.2.6 MediaStreamRecorder

The MediaStreamRecorder needs to be able to handle the case that arises when changes are made to the MediaStreamTrackList objects of the MediaStream being recorded; e.g., a new track is added as a result of the add() method being invoked.

interface MediaStreamRecorder {
    voice getRecordedData (BlobCallback? callBack);
};

3.2.7 BlobCallback

[Callback, NoInterfaceObject]
interface BlobCallback {
    void handleEvent (Blob blob);
};

3.2.8 URL

partial interface URL {
    static DOMString createObjectURL (MediaStream stream);
};

4.1.1 Attributes

iceState of type unsigned short, readonly

The iceState attribute must return the state of the PeerConnection ICE Agent ICE state, represented by a number from the following list:

PeerConnection . NEW (0)

The object was just created, and no networking has yet occurred.

PeerConnection . ICE_GATHERING (0x100)

The ICE Agent is attempting to establish a gather addresses.

PeerConnection . ICE_WAITING (0x200)

The ICE Agent is waiting for candidates from the other side before it can start checking.

PeerConnection . ICE_CHECKING (0x300)

The ICE Agent is checking candidates but has not yet found a connection.

PeerConnection . ICE_CONNECTED (0x400)

The ICE Agent has found a connection but is still checking other candidates to see if there is a better connection.

PeerConnection . ICE_COMPLETED (0x500)

The ICE Agent has finished checking and found a connection.

PeerConnection . ICE_FAILED (0x600)

The ICE Agent is finished checking all candidates and failed to find a connection.

PeerConnection . ICE_CLOSED (0x700)

The ICE Agent has shut down and is no longer responding to STUN requests.

localStreams of type array of MediaStream, readonly

Returns a live array containing the streams that the user agent is currently attempting to transmit to the remote peer (those that were added with addStream()).

Specifically, it must return the read-only MediaStream array that the attribute was set to when the PeerConnection’s constructor ran.

onaddstream of type Function, nullable

This event handler, of event handler event type addstream, must be supported by all objects implementing the PeerConnection interface.

onconnecting of type Function, nullable

This event handler, of event handler event type connecting, must be supported by all objects implementing the PeerConnection interface.

onopen of type Function, nullable

This event handler, of event handler event type open, must be supported by all objects implementing the PeerConnection interface.

onremovestream of type Function, nullable

This event handler, of event handler event type removestream, must be supported by all objects implementing the PeerConnection interface.

onstatechange of type Function, nullable

This event handler, of event handler event type open, must be supported by all objects implementing the PeerConnection interface. It is called any time the readyState, iceState, or sdpState changes.

readyState of type unsigned short, readonly

The readyState attribute must return the PeerConnection object’s PeerConnection readiness state, represented by a number from the following list:

PeerConnection . NEW (0)

The object was just created, and no networking has yet occurred.

PeerConnection . NEGOTIATING (1)

The user agent is attempting to establish an connection with the ICE Agent and to negotiate codecs with the SDP Agent.

PeerConnection . ACTIVE (2)

The ICE Agent has found a connection the SDP Agent has performed a round of codec negotiation. It is possible for whatever media was negotiated to flow.

PeerConnection . CLOSING (4)

The PeerConnection object is terminating all media and is in the process of closing the Ice Agent and SDP Agent.

PeerConnection . CLOSED (3)

The connection is closed.

remoteStreams of type array of MediaStream, readonly

Returns a live array containing the streams that the user agent is currently receiving from the remote peer.

Specifically, it must return the read-only MediaStream array that the attribute was set to when the PeerConnection’s constructor ran.

This array is updated when addstream and removestream events are fired.

sdpState of type unsigned short, readonly

The sdpState attribute must return the state of the PeerConnection SDP Agent , represented by a number from the following list:

PeerConnection . NEW (0)

The object was just created, and no networking has yet occurred.

PeerConnection . SDP_IDLE (0x1000)

At least one SDP offer or answer has been exchange and the SDP Agent is ready to send an SDP offer or receive an SDP answer.

PeerConnection . SDP_WAITING (0x2000)

The SDP Agent has sent and offer and is waiting for a answer.

PeerConnection . SDP_GLARE (0x3000)

The SDP Agent received an offer while waiting for an answer and now much wait a random amount of time before retrying to send the offer.

4.1.2 Methods

addStream

Attempts to starting sending the given stream to the remote peer. The format for the MediaStreamHints objects is currently undefined by the specification.

When the other peer starts sending a stream in this manner, an addstream event is fired at the PeerConnection object.

When the addStream() method is invoked, the user agent must run the following steps:

1. Let stream be the method’s first argument.

2. Let hints be the method’s second argument.

3. If the PeerConnection object’s PeerConnection readiness state is CLOSED (3), throw an INVALID_STATE_ERR exception.

4. If stream is already in the PeerConnection object’s localStreams object, then abort these steps.

5. Add stream to the end of the PeerConnection object’s localStreams object.

6. Return from the method.

7. Parse the hints provided by the application and apply them to the MediaStream, if possible.

8. Have the PeerConnection add a media stream for stream the next time the user agent provides a stable state. Any other pending stream additions and removals must be processed at the same time.

Parameter	Type	Nullable	Optional	Description
stream	MediaStream	✘	✘
hints	MediaStreamHints	✘	✘

Return type: void

close

When the close() method is invoked, the user agent must run the following steps:

1. If the PeerConnection object’s PeerConnection readiness state is CLOSED (3), throw an INVALID_STATE_ERR exception.

2. Destroy the PeerConnection ICE Agent, abruptly ending any active ICE processing and any active streaming, and releasing any relevant resources (e.g. TURN permissions).

3. Set the object’s PeerConnection readiness state to CLOSED (3).

The localStreams and remoteStreams objects remain in the state they were in when the object was closed.

No parameters.

Return type: void

processSignalingMessage

When a message is relayed from the remote peer over the signaling channel is received by the Web application, pass it to the user agent by calling the processSignalingMessage() method.

The order of messages is important. Passing messages to the user agent in a different order than they were generated by the remote peer’s user agent can prevent a successful connection from being established or degrade the connection’s quality if one is established.

When the processSignalingMessage() method is invoked, the user agent must run the following steps:

1. Let message be the method’s argument.

2. Let connection be the PeerConnection object on which the method was invoked.

3. If connection’s PeerConnection readiness state is CLOSED (3), throw an INVALID_STATE_ERR exception.

4. Let header be all characters in message up to the first U+000A LINE FEED (LF) character.

5. Split header on spaces to obtain a list of header components. If the length of list of header components is anything other than 3, then throw a NOT_SUPPORTED_ERR exception and abort these steps.

6. If the first token in list of header components is not a case-sensitive match for "SDP", then throw a NOT_SUPPORTED_ERR exception and abort these steps.

7. Let type be the value of the second token in list of header components.

8. If the third token in list of header components is equal to the PeerConnection id associated with connection, let local be true, otherwise let local be false.

9. Let sdp be all characters in message after the first occurrence of two consecutive U+000A LINE FEED (LF) characters.

10. If type is a case-sensitive match to "OFFER", and local is true, then instruct the PeerConnection SDP Agent to process sdp as a local description of type offer and abort these steps.

11. If type is a case-sensitive match to "OFFER", and local is false, then instruct the PeerConnection SDP Agent to process sdp as a remote description of type offer and abort these steps.

12. If type is a case-sensitive match to "ANSWER", and local is true, then instruct the PeerConnection SDP Agent to process sdp as a local description of type answer and abort these steps.

13. If type is a case-sensitive match to "ANSWER", and local is false, then instruct the PeerConnection SDP Agent to process sdp as a remote description of type answer and abort these steps.

14. If type is a case-sensitive match to "CANDIDATE", and local is false, then let the PeerConnection ICE Agent process sdp and abort these steps.

    JSEP will specify how the SDP and ICE Agents will react upon receiving the signaling messages.

15. Throw a NOT_SUPPORTED_ERR exception.

When a PeerConnection ICE Agent forms a connection to the the far side and enters the state ICE_CONNECTED, the user agent must queue a task that sets the PeerConnection object’s PeerConnection readiness state to ACTIVE (2) and then fires a simple event named open at the PeerConnection object.

Parameter	Type	Nullable	Optional	Description
message	DOMString	✘	✘

Return type: void

removeStream

Stops sending the given stream to the remote peer.

When the other peer stops sending a stream in this manner, a removestream event is fired at the PeerConnection object.

When the removeStream() method is invoked, the user agent must run the following steps:

1. Let stream be the method’s argument.

2. If the PeerConnection object’s PeerConnection readiness state is CLOSED (3), throw an INVALID_STATE_ERR exception.

3. If stream is not in the PeerConnection object’s localStreams object, then abort these steps.

4. Remove stream from the PeerConnection object’s localStreams object.

5. Return from the method.

6. Have the PeerConnectionremove the media stream for stream the next time the user agent provides a stable state. Any other pending stream additions and removals must be processed at the same time.

Parameter	Type	Nullable	Optional	Description
stream	MediaStream	✘	✘

Return type: void

4.1.3 Constants

ACTIVE of type unsigned short

A connection has been formed and if any media streams were successfully negotiated, any relevant media can be streaming.

CLOSED of type unsigned short

The close() method has been invoked.

CLOSING of type unsigned short

The object is starting to shut down after the close() method has been invoked.

ICE_CHECKING of type unsigned short

The ICE Agent is checking candidates but has not yet found a connection that works.

ICE_CLOSED of type unsigned short

The ICE Agent is terminating and will no longer repined to STUN connectivity checks.

ICE_COMPLETED of type unsigned short

The ICE Agent has finished checking all candidates and a connection has been formed.

ICE_CONNECTED of type unsigned short

The ICE Agent has found at least one candidate that works but is still checking.

ICE_FAILED of type unsigned short

The ICE Agent has finished checking all candidates and no connection was worked.

ICE_GATHERING of type unsigned short

The ICE Agent is gather addresses that can be used.

ICE_WAITING of type unsigned short

THE ICE Agent has complete gathering addresses and is waiting for candidates to start checking.

NEGOTIATING of type unsigned short

The peerConenction object is attempting to get to the point wehre media can flow.

NEW of type unsigned short

The object was just created and its ICE and SDP Agent have not yet been started.

SDP_GLARE of type unsigned short

Both side sent SDP offers at the same time and the SDP Agent is waiting to be able to retransmit the SDP offer.

SDP_IDLE of type unsigned short

A valid offer anser pair has been exchanged and the SDP Agent is waiting for the next SDP transaction.

SDP_WAITING of type unsigned short

The SDP Agent has sent an SDP offer and is waiting for a response.

4.2.1 Methods

handleEvent

Def TBD

Parameter	Type	Nullable	Optional	Description
message	DOMString	✘	✘
source	PeerConnection	✘	✘

Return type: void

7.1.1 Attributes

track of type MediaStreamTrack, readonly, nullable

The track attribute represents the MediaStreamTrack object associated with the event.

7.1.2 Dictionary MediaStreamTrackEventInit Members

track of type MediaStreamTrack, nullable

-

7.2.1 Attributes

stream of type MediaStream, readonly, nullable

The stream attribute represents the MediaStream object associated with the event.

7.2.2 Dictionary MediaStreamEventInit Members

stream of type MediaStream, nullable

-