Re: Constraints for resolution: draft-alvestrand-constraints-resolution-00.txt

Let's take a look at the example you provide in your draft below:

getUserMedia({
       video: { mandatory: { minWidth: 1024, minHeight: 768 } }
    }, successCallback, errorCallback);

Let's assume for a moment that this _may_ work only small proportion of 
the time. Once we start applying more constraints that number continues 
to fall exponentially.

What the above code is suggesting is to support an exclusive design 
approach rather than an inclusive design approach - where if a user 
doesn't meet developer-decided expectations they will be flat out 
rejected without further explanation or regard.

My question on your code above is: what if you have successfully 
obtained the HD video above and then you realize that the user is 
actually on a really slow network that would not be capable of streaming 
this HD video?

You're going to want to scale that video down to something more amenable 
to the current internet connection.

What I'd like to therefore see instead is something similar to the 
following:

getUserMedia({ video: true }
    }, successCallback, errorCallback);

function successCallback(stream) {
   // attempt to scale up to HD resolution (if it is supported
   // on this device)
   stream.videoTracks[0].width  = 1024;
   stream.videoTracks[0].height = 768;

   // setup and start streaming the video (out of scope)

}

During the course of streaming, if you want to subsequently downgrade 
the pixel density of the video, at any time, to account for e.g. varying 
connection speeds or e.g. screen-in-screen display, you should be able 
to do something akin to the following:

   stream.videoTracks[0].width  *= .5;
   stream.videoTracks[0].height *= .5;

The great thing about this approach is, as Adam pointed out on the 
recent conference call, that the web page can report in detail why a web 
application is not willing to work with a camera media stream provided 
to that web page. Even though the web browser provided a camera stream 
to the web app as requested the application can decide not to work with 
it if they choose and they can say ('You browser doesn't support HD 
video' or 'You're web camera is not capable of 30 fps', etc).

So for example, if a web application only supports HD video, which I 
would argue is a _very poor_ design approach for something like the web, 
a developer could enforce that behavior in their web application with 
something similar to the following code:

function successCallback(stream) {
   // attempt to scale up to HD resolution (if it is supported
   // on this device)
   stream.videoTracks[0].width  = 1024;
   stream.videoTracks[0].height = 768;

   // this application ONLY supports HD video. Abort if we can't
   // modify the current stream to HD resolution
   if(stream.videoTracks[0].width !== 1024 ||
        stream.videoTracks[0].height !== 768) {
     console.log("You must have a HD Camera to use this app. Aborting.");
     return;
   }

   // setup and start streaming the video (out of scope)

}

Of course, what a conscientious web developer would do is implement the 
first snippet which I'll reproduce here:

function successCallback(stream) {
   // attempt to scale up to HD resolution (if it is supported
   // on this device)
   stream.videoTracks[0].width  = 1024;
   stream.videoTracks[0].height = 768;

   // setup and start streaming the video (out of scope)

}

...which says: I _really_ want to stream HD video if that's a supported 
option. _Otherwise, I'll fallback to whatever resolution the web cam 
supports_.

As we've seen above, a web application can enforce any particular 
capabilities on to a web cam stream to be mandatory but that occurs 
after getUserMedia invocation has already successfully returned a web 
camera stream. The implication here is that we can support all of the 
available use cases. The only difference seems to be one of user 
experience. If a web application only supports HD video resolutions and 
refuses to fallback to any stream of lesser quality then that is to be 
considered a failing of the developer/service.

The side effect of this approach is to make web developers fully aware 
that if they do not intend to work within the capabilities of the 
majority (read: 99%+) of their user's hardware then there will be 
penalties for doing so in the form of having to inform their users that 
they're not willing to work with any provided camera streams because 
they don't meet some pre-determined, idealized capabilities the initial 
developers had in mind.

I'd like to see our efforts moving in this direction away from the 
intentional exclusion of web users with lesser hardware capabilities 
such as those emerging slowly on to the web from e.g. the developing 
world or those with lesser hardware capabilities or those that can't or 
don't keep up with having the latest and greatest gadgets all of the time.

Both inclusive and exclusive types of services remain implementable in 
this approach. Ultimately it is up to developers to decide which 
category they want to be in and implement their apps/services 
accordingly. Market forces will then decide whether those services 
succeed or not.

Web applications that are adaptive and have flexibility built-in to 
their streaming capabilities should win. Exclusionary web apps/services 
that require 'ideal' user hardware and remain inflexible to changing 
user hardware trends and environments should lose.

- Rich

Harald Alvestrand wrote:
>   [technical contributor hat on]
> Since I'm more comfortable with the I-D submissions process than
> anything else, I decided to write an internet-draft detailing a proposal
> for constraints to use for resolution manipulation.
> A copy of the text version attached.
>
> Comments appreciated.
>
> -------- Original Message --------
> Subject:  New Version Notification for
> draft-alvestrand-constraints-resolution-00.txt
> Date:  Sat, 25 Aug 2012 23:03:33 -0700
> From:  internet-drafts@ietf.org
> To:  harald@alvestrand.no
>
>
>
> A new version of I-D, draft-alvestrand-constraints-resolution-00.txt
> has been successfully submitted by Harald Alvestrand and posted to the
> IETF repository.
>
> Filename:  draft-alvestrand-constraints-resolution
> Revision:  00
> Title:   Resolution Constraints in Web Real Time Communications
> Creation date:  2012-08-26
> WG ID:   Individual Submission
> Number of pages: 7
> URL:http://www.ietf.org/internet-drafts/draft-alvestrand-constraints-resolution-00.txt
> Status:http://datatracker.ietf.org/doc/draft-alvestrand-constraints-resolution
> Htmlized:http://tools.ietf.org/html/draft-alvestrand-constraints-resolution-00
>
>
> Abstract:
>     This document specifies the constraints necessary for a Javascript
>     application to successfully indicate to a browser that supports
>     WebRTC what resolutions it desires on a video stream.
>
>
>
>
>
> The IETF Secretariat
>
>
>
>
>

Received on Monday, 27 August 2012 09:54:42 UTC