Re: Re: Synchronous getUserMedia proposal

My general concern here is to avoid creating a programming idiom in
which the "natural" programming idiom leads the programmer thinking
that media is supplied when it is not. Let's take a simple example of
a calling application:

  getUserMedia({audio:true, video:true},
               function(x) {
                 console.log("Success");
                 vidtag.mozSrcObject = x;
                 startCall();
               },
               function() {
                 alert("Audio/video not accessible. Error=" + x);
               }
  );


Now, consider a new API in which we get a promise. Now, there are
two ways to write this code:

1. Pessimistic
  vidtag.mozSrcObject = getUserMedia({audio:true, video:true},
                                     function(x) {
                                       console.log("Success");
                                       startCall();
                                     },
                                     function() {
                                       alert("Audio/video not accessible.
Error=" + x);
                                     }
  );


2. Optimistic:
  vidtag.mozSrcObject = getUserMedia({audio:true, video:true},
                                     function(x) {
                                     },
                                     function() {
                                       alert("Audio/video not accessible.
Error=" + x);
                                     }
  );
  console.log("Success");
  startCall();

The problem is that the second variant is *easier* but bad, since if
the user refuses permission you have this call to someone where you're
just sending blank video [0]. It's especially bad b/c it's tempting to
instead write:

  vidtag.mozSrcObject = getUserMedia({audio:true, video:true});
  console.log("Success");
  startCall();

We certainly know that users routinely ignore error return codes in
regular C/C++ code, so I don't think it's at all weird to think that
they would write this kind of code.

My general policy here is that we should design APIs that encourage
safe programming idioms, not ones that encourage unsafe programming
idioms. So, my concern with this design is that it does the opposite:
if the user doesn't explicitly check for errors, he is left in
a funky state. Worse yet, this is something that will work in his
lab because he will approve permissions.


There appear to be two primary motivations for this functionality:

- To have a "temporary" stream to provide to the PC to get a
  head start on negotiation.
- To allow self/preview video before permission is granted.

The first of these seems problematic because we don't know what
device the user will select. So, for instance, if I have two
cameras, one of which has an H.264 encoder and one does not,
should I offer H.264? What happens if the user selects another
camera. Moreover, if you're concerned with fingerprinting
(I'm mostly) the offer seems to reveal a lot of information
about the devices. Are we good with that?

WRT to the second question: is this a feature that people think
is important? I think I could go either way....

IMO we should determine whether these are actually important
features before we rework the API.
-Ekr


[0] Full disclosure: it's still possible to make this kind of error
with the existing API. Try loading up apprtc.appspot.com in tab A
and then refusing camera access. Then load up apprtc.appspot.com
in Tab B and allow access






On Mon, Nov 19, 2012 at 5:20 AM, Harald Alvestrand <harald@alvestrand.no>wrote:

>  Moving this discussion to media-capture list, since that's where it
> belongs.
>
>
> -------- Original Message --------  Subject: Re: Synchronous getUserMedia
> proposal  Resent-Date: Mon, 19 Nov 2012 13:13:41 +0000  Resent-From:
> public-webrtc@w3.org  Date: Mon, 19 Nov 2012 14:13:08 +0100  From: Harald
> Alvestrand <harald@alvestrand.no> <harald@alvestrand.no>  To:
> public-webrtc@w3.org
>
>
> Entering the stream early, even though the date is late...
>
> there was an alternate proposal being floated in Lyon.
>
> MediaStreamEventThingy getUserMedia(MediaConstraints constraints,
> optional SuccessCallback, optional ErrorCallback)
>
> interface MediaStreamEventThingy : EventTarget {
>    MediaStream theStream;  // in state "muted" until success
>    attribute EventHandler onsuccess;
>    attribute EventHandler onfailure;
> }
>
> if SuccessCallback is given, this is 100% the same as setting onsuccess
> to function(e) { callback(e.stream) }, and similarly for failure. This
> lets existing code go on working.
>
> This (a result object + onsuccess / onfailure handlers) is a pattern
> that I've observed elsewhere (in IndexedDB, for instance). Not being
> creative is a Good Thing.
>
> Note: People who want to shoot themselves in the foot will do
>
> videoTag.srcObject = GetUserMedia(constraints).theStream
>
> but this is incrementally more difficult than the "simply return a
> stream" option, so it's at least no worse.
>
> On 11/05/2012 11:41 AM, Adam Bergkvist wrote:
> > On 2012-11-02 19:32, Martin Thomson wrote:
> >> In its simplest form:
> >>
> >> MediaStream getUserMedia(MediaConstraints constraints);
> >>
> >> This returns a stream that provides no content (open option: a tainted
> >> stream that can only be displayed locally).
> >>
> >> Consent is indicated with a new onconsent event on the stream; failure
> >> reuses the onended event.  A new reason parameter is added to the
> >> onended event indicating the reason (this includes all existing onended
> >> reason codes, if any, plus all getUserMedia error codes).
> >>
> >> The major complaint with this is that it leads to an
> >> inaccurate/misleading expectation about the usability of the stream.
> >> That expectation can lead to the assumption that consent is granted,
> >> which would be a bad assumption.
> >
> > This approach is not flawless, but to me it seems like the most
> > reasonable one at the moment.
> >
> > We already have the concept of a stream that is dispatched to
> > JavaScript but the source is not ready to provide data yet. This
> > currently happens when you receive a stream over a PeerConnection and
> > all the tracks are muted until data arrives over the network. I think
> > gUM() with a return value could be treated similarly, and local data
> > is suspended until the user grants permission.
> >
> > In the network case, a media description is used to create the stream
> > and the receiving side and it's pretty capable of describing future
> > stream content. In our local case, the user may only grant one media
> > component. Perhaps ended track state is good enough to solve this.
> >
> > I think we'll freak people out if a tainted stream is delivered at
> > once. Even though page authors can't access the content or transport
> > the stream, they can mix the camera view into the page content and
> > that may make people uncomfortable (depending on the page they're
> > visiting).
> >
> > /Adam
> >
>
>
>
>
>
>
>