Re: Synchronous getUserMedia proposal

On 14 November 2012 14:20, Travis Leithead
<travis.leithead@microsoft.com> wrote:
> I am surprised that I haven't heard much more pushback on this design approach. I suppose that means it's an inevitable transition.

Don't sound so fatalistic. :)

> A few questions:
> 1. If the user agent doesn't have any cameras what happens? (Perhaps a null value is returned? A fake Media Stream in the ENDED state?) Generally speaking, what do we do with all the old error conditions?

My thought was that all error conditions become manifest in the
"ended" event on the stream.  That includes: no camera, user denies
permission, etc...

> 2. How are multiple cameras handled supported? By multiple calls to the API as before? It seems like this aspect of the old design needs to change.

The old design was a little strange.  Multiple calls to getUserMedia
would return different cameras if they were called together (prior to
reaching stable state).  Was there any specific expectation around
what would happen in subsequent calls otherwise?

It seems like it would be reasonable to offer some amount of control over this:
 - constraint for selecting a specific source
 - constraint for selecting a new source
Source identities might have to be obtained through live streams
within the current browsing context.  I'd be concerned if the
identifier were persistent, or discoverable without user consent.

> An alternative idea is to use getUserMedia as an approval/activation method for track "promises". As such you'd need a way to create appropriate track "placeholders" and getUserMedia would "upgrade" these to actual media-containing tracks. Consider:
> var videoPlaceholder = new MediaStreamTrack("video");
> var audioPlaceholder = new MediaStreamTrack("audio");
> var placeholderMS = new MediaStream([videoPlaceholder, audioPlaceholder]);
>
> The above objects are in a "not started" state and are not tied to any source [yet]. Then getUserMedia will try to bind all the placeholder tracks to real media sources and may succeed, fail, or partially succeed, given the number of requested placeholder tracks. Reporting for each binding failure/success will be in the form of events on each respective track placeholder.

This could work as long as the placeholder streams were each given an
identifier that is usable and unchanging.

> void getUserMedia(MediaConstraints constraints, MediaStream unlockMyTracks);
>
> The advantages: getUserMedia will know at invocation-time how many media sources to try and activate and can show the user a suitably relevant UI only once--the user stays in control of activating media devices, and can do so at their leisure--it doesn't impact the use of the MediaStream and/or track objects.
>
> Disadvantages: in-place upgrading of tracks from generic MediaStreamTrack types to derived VideoDeviceTrack, etc., might be a little weird since this "upgrade" would suddenly expose new APIs that didn't exist before on the MediaStreamTrack. This is another argument against the current Track inheritance model vs. a composition-based model.

I like this idea, at least as much as the synchronous proposal.  Maybe
it's better to characterize this as "attach MediaStream to source"
rather than "placeholder".  Maybe constructing the specific class is
possible:

var video = new VideoTrack();
var audio = new AudioTrack();
var stream = new MediaStream([audio, video]);
navigator.getUserMedia(constraints, stream, successCB, failureCB);

Composition might be more appropriate for the picture capture options
on video, but I think that asking for specific specializations of
MediaStreamTrack is not a terrible choice.

Received on Thursday, 15 November 2012 20:53:18 UTC