Re: Direction: Constraints

Cullen, my responses are in-line for your consideration.

Cullen Jennings wrote:
> On May 17, 2012, at 2:26 PM, Rich Tibbett wrote:
>
>> On May 17, 2012, at 9:03 PM, Cullen Jennings<fluffy@iii.ca>  wrote:
>>
>>> On May 15, 2012, at 3:46 AM, Rich Tibbett wrote:
>>>
>>>> Opera cannot integrate this in to our current products since we are not yet pursuing a trusted environment model for the web on which this proposal heavily relies.
>>> First - this makes no sense to me in two different ways - the constraints proposal does not rely on a trusted environment. Second, we might mean different things by a trusted environment but I read this to say you are going to allow access to the camera with no user authorization. I seriously doubt this is what you mean so I suspect what we should be  talking about is what sort of trust we are is needed for a minimal implementation.
>> We will of course be providing user opt-in to camera access [1].
>>
>> It is not my understanding that providing user authorisation for a web page to access the camera is the same thing as handing over the ability to leak fingerprinting information
>
> Uh, now the issue seems to be a privacy issue. I'm lost.

How is privacy not an aspect of user trust? Does a user trust things 
that leak their privacy or are the two not, in fact, the same thing?

This is a trust issue as I said above. We often don't have a way on the 
web to e.g. resolve a web page back to its original author and therefore 
have an avenue to pursue for legal litigation if something goes wrong. 
That is a genuine safe-guard in the app store model and that is a 
fundamental separation between what the web provides and what a curated 
app store can provide. A curated app store is the only place that we 
would be comfortable to provide an API with such serious potential 
side-effects because there is a certain level of developer 
accountability implied in that environment.

> Clearly any API that delivers the largest size image the camera is capable of will end up revealed some fingerprinting information. As soon as you provide an actual image, you provide access to the see how the firmware did the white balance, auto gain, and in some cases how the bad pixels in the CCD were filled in along with the location of the band pixels in the image. That's pretty brutal from a finger printing point of view and bad pixels location more or less uniquely identifies the camera. Even for cameras where one can't find the bad pixel location, the issues of what values the camera tried to set the auto gain and white balance for often leak a ton of information about what type of camera was used.

That's certainly true and an interesting consideration we need to 
discuss further.

That doesn't give us carte blanche to completely give up the user's 
'identity' via further privacy leakages since we've sprung a minor 
privacy leak already.

>
> Now, I'm sure we need a long conversation about the fingerprinting of all all these API at some point and that will impact what parts of the API can be used at which point. But I don't see how that has to do with the basic idea of the constraints proposal.

This is surely a classic case of putting the cart before the horse: we 
will have a fixed solution regardless of any issues we may come up with. 
It should clearly be the other way around: we will design a solution 
when we have identified a number of issues of which a number have 
already remain unanswered on this list.

>
>> or give developers the nightmare task of working out a meaningful subset of complex constraints to target their applications at.
>> User authorisation to access the camera should not come with the need to hand over my 'identity' or give sites the ability to track me across sessions.
>
> sure - I agree that is one use case. Again, don't see how that makes constraints a bad idea.
>
>> Accessing the camera should not require deep internal knowledge of codecs or their parameters (not to mention ensuring I'm using the right combinations, names, units and scales).
>
> total agree and the proposal does not do this

It gives web developers the ability to do this and they will invariably 
get it wrong by being too narrow on their criteria, setting the 
threshold for their app too high or completely misunderstanding and 
misrepresenting the actual capabilities of their user's hardware and 
software.

>
>> As a simple thought experiment: I could be identified in a private tab despite not being logged in had I previously logged in to the service in a non private tab at any prior point based on the unique combination of constraints created by my current hardware, OS, webcam type, drivers installed and user preferences. This group just doesn't seem to have evaluated those aspects of the proposal.
>
> Though I agree that more thought is needed on that, I think the WebRTC WG has discussed theses at many points and raised many issues. The privacy issues are huge with accessing a camera - work is proceeding with that, but, as much as I know it will pain you, me, and Anant, all of the access to microphones and cameras and codecs will make things worse to varying degrees - we will need to decide what the threat model is, and what tradeoffs we want to make. Not using constrains does not change the work that needs to be done with respect to this.

Agreed. Using constraints does increase the work required on this though 
so it has to be an absolutely necessary addition. I hold constraints to 
an even higher standard of actual need than other parts of this API due 
to the privacy and trust concerns.

>
>> As a secondary example: as a developer I set my web app to only work with cameras capable of providing a resolution above 1024x768 that provide 60+ frames per second. I have just excluded ~90% of users from even trying to use my web app. Why did I choose these values?
>
> Clearly because that is what the application needed.
>
>> What about the developing world with inferior hardware and software that didn't even cross my mind? Do I _really_ need these constraints for my web app to work or could we just try to work with what the users have actually got?
>
> This question is verging on lame - if the developer set them as mandatory, clearly there was a need.
>
>> In reality lesser values would have worked
>
> Really ? if that was the case the developer would have set them as optional not mandatory. An application is welcome to set nothing in the constraints in which case the API will be as you suggestion. However, many developers have made it very clear that they need more control than this.

You're giving web developers way too much credit for knowing what 
they're doing in the last couple of replies :)

Web developers can, and often already do, get much much simpler things 
than this wrong in web design e.g. 
http://webdesignledger.com/tips/the-most-common-html-and-css-mistakes-to-avoid

Funnily enough, even a number of the experts on this mailing list, have 
gotten it wrong while writing their constraints examples out to this 
list. They've used the wrong units, the wrong combinations of 
constraints, they've broken the rules of Javascript variable naming and 
they've missed commas and quote marks at various intervals. _The 
experts_. _In this expert group_ :)

>
>> just fine but I didn't add them because I couldn't possibly begin to imagine where to begin with that or where to set the threshold. Instead I just wish the user agent would give me a video stream that will work everywhere or just give me the 'best stream you've got' and we'll go from there. If my e.g. Image recognition doesn't work at the point of actually trying it then I can honestly say that it wasn't because we accidentally excluded 90% of the web before they even got to try it.
>>
>>>> The upshot of this is that we will not be able to claim conformance to the getUserMedia specification if this is added.
>>>> This and the fact that there is a lot of hidden complexity and a lot of unresolved issues in this proposal
>>> Really? more complex than what? This is the problem: many of us can not figure out what it is you want - all we know if you don't like anything that is proposed. So what is it exactly that you think this is more complex than. And what requirements do you think we need to meet - for example, most the people I have talked to think that we need to be able to indicate some hints about the desired resolution of image capture.
>> This is more complex than _nothing at all at the point of media stream acquisition_ which has proved capable of handling all of our tried and tested use cases to date.
>
> The WebRTC WG seemed pretty clear on they needed more than the nothing you have today. Obviously the chairs can fill you in on past meetings but rooms with a lot of people in them seemed pretty unanimous in this.
>
>> In the course of prototyping on our implementation of this over the last 15 months, not a single developer has requested the features proposed by constraints.
>
>> Not a single use case purporting to its need has yet to stand up to much scrutiny either. _If someone has a use case for constraints *at getUserMedia acquisition* that stands up to scrutiny then it would be very helpful.
>
> This has been discussed to death - we have a ton of use cases. Your answer to all of them is you don't accept that use case. It's not a discussion I can really figure out how to engage in.

Where are these use cases documented? If they don't apply to 
non-recording, non-streaming use cases then they don't apply at the 
point of getUserMedia invocation but further down the tool chain.

>
>> My point is (and has been) that this is the wrong point to be asking about resolutions or frame rates or the like. Those characteristics imply a secondary usage such as recording or streaming and should be requested independently at those points. Constraints should live at those points (if anywhere). An acquired media stream will give you whatever the platform is capable of providing which you can further manipulate downstream
>
> Lots of USB camera can not simultaneously provide full resolution and full frame rate. Only you application knows which it would prefer. If you don't reply to anything else in this email, please think about this and let me know what you would propose.

I'm fairly sure you've just proven my point. If I make both full 
resolution and full frame rate mandatory then I've excluded ~90%+ of 
people from using my web site. As a web developer I'm expected to know 
the general rule you've cited?

Again you're giving web developers way to much credit for the in-depth 
knowledge they require to correctly use something with this many moving 
parts.

>
>> On the contrary at the point you obtain a media stream there are some (very few) meta characteristics that you _do_ want to request but nothing to do with the current proposal. Right now we're looking for a simple way for a page to request the front or back-facing camera. That is not a mandatory request since either camera will work but in some cases while a developer may know best which camera type is best suited for their application, and a user agent will do it's best to suggest the user to authorise the requested camera, a user may want to, and should be able to, choose the other camera.
>
>>>> is likely to significantly delay delivery of the core getUserMedia specification in the standards process.
>>>>
>>>> It would be preferable if trusted environment features are pursued in a separate specifcation for now and the open issues on this proposal, of which there are many, are pursued orthogonally to the main task of nailing down the core features of the getUserMedia specification.
>> [1] http://dev.opera.com/articles/view/getusermedia-access-camera-privacy-ui/
>
> Look, what we are proposing here would allow you to do everything that API does with JS function calls that are seem pretty much about the same as that API plus a few more curly, round and square braces here and there. If opera chooses to reject all mandatory constraints and ignore all optional constraints, it seems like the implementation will be about the same too.
>

Providing a standard interface with an optional advanced interfaces does 
not work on the web. People invariably use advanced interfaces entirely 
innocently but completely erroneously, without understanding the full 
consequences of their choices in potentially excluding a large 
proportion of their user base.

I've scratched the surface at this point wrt to potential issues with 
any type of constraints-like proposal. The issue is not with this 
specific proposal. The issue is with the underlying principles of adding 
anything like this to the web unless it is something that is absolutely 
essential. Invariably it is not.

Received on Friday, 18 May 2012 15:36:35 UTC