Re: Capabilities API proposal

On Jan 24, 2012, at 3:59 PM, Randell Jesup wrote:

> On 1/24/2012 6:05 AM, Dan Burnett wrote:
>> I'm going to be replying to some other emails shortly, including Cullen's comments, but before I do I think it would be helpful for me to explain the thinking that led to my API proposal.
>> 
>> When we first began discussing this at Voxeo, as a thought experiment we began with the premise that a well-designed Hints API would make a Capabilities API unnecessary.  After all, if a web browser could sufficiently understand what an app author wanted to do, then there would be no need to query capabilities.
>> We immediately ran into two problems with this.  First, this did not allow us to grey out or hide options known to be unavailable.  For example, if we knew that a camera capable of generating a decent resolution image was not available we would not want to even show that as an option in the UI.  Clearly this has privacy implications and should be discussed in the larger context of how much capability information should be available to the application (possibly differing based on whether permission has been given).  Second, we also realized that in many cases we had minimum requirements.  For example, if I only want HD conferencing, I don't want to request that in a hint and be given 640x480 greyscale.
>> Our next thought was to give hints but, for each hint, also give a minimum that we would accept.  The problem with this is two-fold:  first, we would need to add additional syntax to differentiate between what we would like (the hint) and what we would accept (the minimum), and second, this doesn't address the either-or case.  An example of the latter would be wanting 1080p video unless it wasn't available, in which case you wanted only a thumbnail video.
> 
> All good points.
> 
>> At this point we realized that application authors will *always* want control over what they get.  Since we will be using a hints API that does not guarantee you will get exactly what you request, the next best option is to be able to specify configurations/profiles that would be acceptable, such that the browser is expected to give you any *one* of the configurations you would be happy with.  To increase the chances that the browser will give you something close to what you request and, most importantly, a configuration with options that are internally consistent with respect to your application, without a low-level API the easiest way to achieve this is to have the browser itself tell you in advance (via getCapabilities) exactly which configurations/profiles of hints it can understand and/or provide.
>> 
>> This led us to propose that 1) the return from getCapabilities() should be a collection of profiles/configurations that the browser is capable of providing, and that 2) the input to the Hints API should be an author-ordered subset of these configurations.  While it does not guarantee in advance which configuration the author will get, it does guarantee that if an acceptable configuration can be provided it will be, without having to try, fail, try again, fail again, etc.
> 
> A problem I see with this is a combinatorial problem: if getCapabilities returns the "collection of profiles/configurations the browser is capable of providing", then when you add multiple devices (multiple mics, multiple cameras, etc) the set of options becomes quite large, and selecting from it becomes more involved.  I'm not saying impossible, but the added complexity for the typical application (and how likely they are to test it properly against possible capabilities) worries me a bit.

"Exercise left for the reader" :)
Seriously, though, I was very aware of that when I proposed this, and I agree it's a problem.  What I'm hoping is that we can have a discussion on how to get the same effect without the combinatorial explosion.  I couldn't come up with a way but was not willing, yet, to drop the ability for an app author to have choice and a modicum of control over what he might actually get for his media stream.
We may well be able to disconnect audio and video, for example.  Where I get uncomfortable is when, for video, we disconnect width, height, aspect ratio, and bandwidth/loss rate as Cullen has proposed.  These are all intimately tied together and may have different relative priorities for different applications.

Do you have any thoughts as to how we might address the combinatorial explosion while retaining good author-level control?

> 
>> 
>> A side benefit of this approach is that if a browser becomes able to provide some new capability, by returning that in the getCapabilities() call it becomes available to the app author.
>> 
>> 
>> The specific examples we show below are not in any way intended to specify the Hints API, but rather to show what a JSON array of profiles/configurations might look like.
>> 
>> -- dan
>> 
>> 
> 
> 
> -- 
> Randell Jesup
> randell-ietf@jesup.org
> 
>

Received on Wednesday, 25 January 2012 13:40:38 UTC