New API surface - inbound outbound streams/tracks

Hi

A while back I sent out a proposal [1] on API additions to represent 
streams that are sent and received via a PeerConnection. The main goal 
was to have a natural API surface for the new functionality we're 
defining (e.g. Stats and DTMF). I didn't get any feedback on the list, 
but I did get some offline.

I've updated the proposal to match v4 of Travis' settings proposal [2] 
and would like to run it via the list again.

Summary of the main design goals:
- Have a way to represent a stream instance (witch tracks) that are sent 
(or received) over a specific PeerConnection. Specifically, if the same 
stream is sent via several PeerConnection objects, the sent stream is 
represented by different "outbound streams" to provide fine grained 
control over the different transmissions.

- Avoid cluttering PeerConnection with a lot of new API that really 
belongs on stream (and track) level but isn't applicable for the local 
only case. The representations of sent and received streams and tracks 
(inbound and outbound) provides the more precise API surface that we 
need for several of the APIs we're specifying right now as well as 
future APIs of the same kind.

Here are the object structure (new objects are marked with *new*). Find 
examples below.

AbstractMediaStream *new*
|
+- MediaStream
|   * WritableMediaStreamTrackList (audioTracks)
|   * WritableMediaStreamTrackList (videoTracks)
|
+- PeerConnectionMediaStream *new*
     // represents inbound and outbound streams (we could use
     // separate types if more flexibility is required)
     * MediaStreamTrackList (audioTracks)
     * MediaStreamTrackList (videoTracks)

MediaStreamTrack
|
+- VideoStreamTrack
|  |
|  +- VideoDeviceTrack
|  |   * PictureDevice
|  |
|  +- InboundVideoTrack *new*
|  |   // inbound video stats
|  |
|  +- OutboundVideoTrack *new*
|      // control outgoing bandwidth, priority, ...
|      // outbound video stats
|      // enable/disable outgoing (?)
|
+- AudioStreamTrack
    |
    +- AudioDeviceTrack
    |
    +- InboundAudioStreamTrack *new*
    |   // receive DTMF (?)
    |   // inbound audio stats
    |
    +- OutboundAudioStreamTrack *new*
        // send DTMF
        // control outgoing bandwidth, priority, ...
        // outbound audio stats
        // enable/disable outgoing (?)

=== Examples ===

// 1. ***** Send DTMF *****

pc.addStream(stream);
// ...

var outboundStream = pc.localStreams.getStreamById(stream.id);
var outboundAudio = outboundStream.audioTracks[0]; // pending syntax

if (outboundAudio.canSendDTMF)
     outboundAudio.insertTones("123", ...);


// 2. ***** Control outgoing media with constraints *****

// the way of setting constraints in this example is based on Travis'
// proposal (v4) combined with some points from Randell's bug 18561 [3]

var speakerStream; // speaker audio and video
var slidesStream; // video of slides

pc.addStream(speakerStream);
pc.addStream(slidesStream);
// ...

var outboundSpeakerStream = pc.localStreams
         .getStreamById(speakerStream.id);
var speakerAudio = outboundSpeakerStream.audioTracks[0];
var speakerVideo = outboundSpeakerStream.videoTracks[0];

speakerAudio.priority.request("very-high");
speakerAudio.bitrate.request({ "min": 30, "max": 120,
                                "thresholdToNotify": 10 });
speakerAudio.bitrate.onchange = speakerAudioBitrateChanged;
speakerAudio.onconstraintserror = failureToComply;

speakerVideo.priority.request("medium");
speakerVideo.bitrate.request({ "min": 500, "max": 1000,
                                "thresholdToNotify": 100 });
speakerAudio.bitrate.onchange = speakerVideoBitrateChanged;
speakerVideo.onconstraintserror = failureToComply;

var outboundSlidesStream = pc.localStreams
         .getStreamById(slidesStream.id);
var slidesVideo = outboundSlidesStream.videoTracks[0];

slidesVideo.priority.request("high");
slidesVideo.bitrate.request({ "min": 600, "max": 800,
                               "thresholdToNotify": 50 });
slidesVideo.bitrate.onchange = slidesVideoBitrateChanged;
slidesVideo.onconstraintserror = failureToComply;


// 3. ***** Enable/disable on outbound tracks *****

// send same stream to two different peers
pcA.addStream(stream);
pcB.addStream(stream);
// ...

// retrieve the *different* outbound streams
var streamToA = pcA.localStreams.getStreamById(stream.id);
var streamToB = pcB.localStreams.getStreamById(stream.id);

// disable video to A and disable audio to B
streamToA.videoTracks[0].enabled = false;
streamToA.audioTracks[0].enabled = false;

======

Please comment and don't hesitate to ask if things are unclear.

/Adam

----
[1] http://lists.w3.org/Archives/Public/public-webrtc/2012Sep/0025.html
[2] 
http://dvcs.w3.org/hg/dap/raw-file/tip/media-stream-capture/proposals/SettingsAPI_proposal_v4.html
[3] https://www.w3.org/Bugs/Public/show_bug.cgi?id=15861

Received on Friday, 9 November 2012 12:14:53 UTC