Re: Connecting devices and tracks

On 08/11/13 20:48, Martin Thomson wrote:
> I was asked to write this up and I've sat on this for a while, worried
> that the barrage of detail would destroy the message.  There's a fair
> degree of nuance in the proposal, so I recommend reading through,
> maybe skipping the code for getUserMedia() and then we can discuss the
> various issues that you see.
>
> # Connecting Devices and Tracks
>
> The device enumeration function provided by getMediaDevices() is
> critical to this proposal.  The object that is produced by
> getMediaDevices(), MediaDeviceInfo, is extended to include a way to
> open and use that source (we can entertain a name change later; it
> doesn't appear in the API explicitly, but it might be nice for a
> better handle):
>
>    partial interface MediaDeviceInfo {
>      MediaStreamTrack createTrack(Constraints? constraints);
>      void connectTrack(MediaStreamTrack track);
>    }
>
> For someone who was going to be using the sourceId constraint to
> select a source, this is really easy to use.
>
>    var track = deviceInfo.createTrack({ ... constraints ... });
>
> It gets a little harder when you don't want to choose, and for that
> purpose I'm going to propose that we retain getUserMedia
> (implementation below).
>
> The constructor of MediaStreamTrack becomes more critical to the
> process.  The track that is acquired as a result of calling the
> constructor directly produces black or silence.  You'll notice that
> this is intentionally synchronous; there are consequences for that,
> but I think that this is something that the browser will have to
> handle.
>
> Any user consent interactions are triggered when connectTrack() is
> called.  Once an active (i.e., unmuted) device is attached to the
> track, the 'onunmuted' event is fired for that track.
>
> createTrack() is a shortcut for both creating a track and connecting it:
>
>    MediaDeviceInfo.createTrack = function createTrack(constraints) {
>      var track = new {Video,Audio}MediaStreamTrack(constraints);
>      this.connectTrack(track);
>      return track;
>    };

Let's see if I've understood correctly:

* If you do createTrack, the UA will immediately create a track 
connected to the source in question, but it will only produce 
silence/blackness until the user approves its use, right? So it could be 
used immediately for negotiation?

Is that about right?

* What is then the use of the MediaStreamTrack constructor? It has been 
argued that if you just construct a track (with no info at all about an 
eventual source) it can't be used for negotiation, and I can see no 
other use for it?


* Is connectTrack supposed to give an error if the track you supply 
already has a source, or is the plan to allow for changing source of a 
track?

I'm trying to understand what this proposal (which I like BTW, but if we 
can get support the same functionality with fewer changes that feels 
safer) buys us over the one Jan-Ivar did a long time ago [1].



[1] 
http://lists.w3.org/Archives/Public/public-media-capture/2013Sep/0080.html

>
> And that's it.
>
> The following code demonstrates how this can be used to replicate the
> behaviour of getUserMedia.  It uses createTrack().
>
> --
>
> navigator.getUserMedia  = function getUserMedia(constraints,
> successCb, failureCb) {
>    // error checking omitted
>    if (constraints.video) {
>      getTrack("video", constraints.video, trackConnected, trackFailure);
>    }
>    if (constraints.audio) {
>      getTrack("audio", constraints.audio, trackConnected, trackFailure);
>    }
>
>    var firstTrack;
>    function trackConnected(track) {
>      if (firstTrack || !constraints.video || !constraints.audio) {
>         successCb(new MediaStream([firstTrack, track]));
>      }
>    }
>
>    var failedAlready;
>    function trackFailure(e) {
>      if (!failedAlready) {
>        failedAlready = true;
>        failureCb(e);
>      }
>    }
> };
>
> function getTrack(type, constraints, successCb, failureCb) {
>    var devices = navigator.getMediaDevices().filter(ofType(type));
>
>    function stopMonitoring(track) {
>      track.removeEventListener('unmute', connected); // overly cautious here
>      track.removeEventListener('overconstrained', tryDevice);
>    }
>
>    function tryDevice(e) {
>      if (e) {
>        if (timeout) {
>          window.clearTimeout(timeout);
>        }
>        stopMonitoring(e.target);
>        e.target.stop();
>      }
>
>      var dev = devices.shift(); // pick the next one
>      if (!dev) {
>        failureCb(new Error('overconstrained'));  // use real thing
>      }
>      var track = dev.openTrack(typeof constraints === 'object' ?
> constraints : undefined);
>      track.addEventListener('unmute', connected);
>      track.addEventListener('overconstrained', tryDevice);
>
>      var timeout = window.setTimeout(tryDevice, 1000);
>    }
>
>    function connected(e) {
>      stopMonitoring(e.target);
>      successCb(e.target);
>    }
> }
>
> function ofType(t) {
>    return function(x) {
>      return x.type == "video";
>    };
> }
>
> --
>
> Please excuse the inevitable bugs and syntax errors; obviously I can't
> compile or run this and I didn't bother linting it.
>
> Obviously, this isn't going to be how a browser would really implement
> this.  This results in the camera going on all cameras in turn, which
> is not ideal.  It might also find a device that meets the provided
> contraints but is suboptimal. Another problem is that this requires a
> timeout in case a muted device is selected because we don't
> distinguish between muted and no user consent (intentionally).  A
> browser can, of course, use the extra knowledge it has about sources
> in order to implement these functions.
>
> Names are, of course, negotiable, as is every other aspect of this.
> For instance, I was considering the use of an overload on
> "connectTrack", but the difference in method signatures seemed enough
> to justify two names.
>
>

Received on Saturday, 9 November 2013 14:55:31 UTC