Connecting devices and tracks

I was asked to write this up and I've sat on this for a while, worried
that the barrage of detail would destroy the message.  There's a fair
degree of nuance in the proposal, so I recommend reading through,
maybe skipping the code for getUserMedia() and then we can discuss the
various issues that you see.

# Connecting Devices and Tracks

The device enumeration function provided by getMediaDevices() is
critical to this proposal.  The object that is produced by
getMediaDevices(), MediaDeviceInfo, is extended to include a way to
open and use that source (we can entertain a name change later; it
doesn't appear in the API explicitly, but it might be nice for a
better handle):

  partial interface MediaDeviceInfo {
    MediaStreamTrack createTrack(Constraints? constraints);
    void connectTrack(MediaStreamTrack track);
  }

For someone who was going to be using the sourceId constraint to
select a source, this is really easy to use.

  var track = deviceInfo.createTrack({ ... constraints ... });

It gets a little harder when you don't want to choose, and for that
purpose I'm going to propose that we retain getUserMedia
(implementation below).

The constructor of MediaStreamTrack becomes more critical to the
process.  The track that is acquired as a result of calling the
constructor directly produces black or silence.  You'll notice that
this is intentionally synchronous; there are consequences for that,
but I think that this is something that the browser will have to
handle.

Any user consent interactions are triggered when connectTrack() is
called.  Once an active (i.e., unmuted) device is attached to the
track, the 'onunmuted' event is fired for that track.

createTrack() is a shortcut for both creating a track and connecting it:

  MediaDeviceInfo.createTrack = function createTrack(constraints) {
    var track = new {Video,Audio}MediaStreamTrack(constraints);
    this.connectTrack(track);
    return track;
  };

And that's it.

The following code demonstrates how this can be used to replicate the
behaviour of getUserMedia.  It uses createTrack().

--

navigator.getUserMedia  = function getUserMedia(constraints,
successCb, failureCb) {
  // error checking omitted
  if (constraints.video) {
    getTrack("video", constraints.video, trackConnected, trackFailure);
  }
  if (constraints.audio) {
    getTrack("audio", constraints.audio, trackConnected, trackFailure);
  }

  var firstTrack;
  function trackConnected(track) {
    if (firstTrack || !constraints.video || !constraints.audio) {
       successCb(new MediaStream([firstTrack, track]));
    }
  }

  var failedAlready;
  function trackFailure(e) {
    if (!failedAlready) {
      failedAlready = true;
      failureCb(e);
    }
  }
};

function getTrack(type, constraints, successCb, failureCb) {
  var devices = navigator.getMediaDevices().filter(ofType(type));

  function stopMonitoring(track) {
    track.removeEventListener('unmute', connected); // overly cautious here
    track.removeEventListener('overconstrained', tryDevice);
  }

  function tryDevice(e) {
    if (e) {
      if (timeout) {
        window.clearTimeout(timeout);
      }
      stopMonitoring(e.target);
      e.target.stop();
    }

    var dev = devices.shift(); // pick the next one
    if (!dev) {
      failureCb(new Error('overconstrained'));  // use real thing
    }
    var track = dev.openTrack(typeof constraints === 'object' ?
constraints : undefined);
    track.addEventListener('unmute', connected);
    track.addEventListener('overconstrained', tryDevice);

    var timeout = window.setTimeout(tryDevice, 1000);
  }

  function connected(e) {
    stopMonitoring(e.target);
    successCb(e.target);
  }
}

function ofType(t) {
  return function(x) {
    return x.type == "video";
  };
}

--

Please excuse the inevitable bugs and syntax errors; obviously I can't
compile or run this and I didn't bother linting it.

Obviously, this isn't going to be how a browser would really implement
this.  This results in the camera going on all cameras in turn, which
is not ideal.  It might also find a device that meets the provided
contraints but is suboptimal. Another problem is that this requires a
timeout in case a muted device is selected because we don't
distinguish between muted and no user consent (intentionally).  A
browser can, of course, use the extra knowledge it has about sources
in order to implement these functions.

Names are, of course, negotiable, as is every other aspect of this.
For instance, I was considering the use of an overload on
"connectTrack", but the difference in method signatures seemed enough
to justify two names.

Received on Friday, 8 November 2013 19:47:54 UTC