Re: Direction: Constraints

On May 17, 2012, at 2:26 PM, Rich Tibbett wrote:

> On May 17, 2012, at 9:03 PM, Cullen Jennings <fluffy@iii.ca> wrote:
> 
>> On May 15, 2012, at 3:46 AM, Rich Tibbett wrote:
>> 
>>> Opera cannot integrate this in to our current products since we are not yet pursuing a trusted environment model for the web on which this proposal heavily relies.
>> 
>> First - this makes no sense to me in two different ways - the constraints proposal does not rely on a trusted environment. Second, we might mean different things by a trusted environment but I read this to say you are going to allow access to the camera with no user authorization. I seriously doubt this is what you mean so I suspect what we should be  talking about is what sort of trust we are is needed for a minimal implementation.
> 
> We will of course be providing user opt-in to camera access [1].
> 
> It is not my understanding that providing user authorisation for a web page to access the camera is the same thing as handing over the ability to leak fingerprinting information

Uh, now the issue seems to be a privacy issue. I'm lost. Clearly any API that delivers the largest size image the camera is capable of will end up revealed some fingerprinting information. As soon as you provide an actual image, you provide access to the see how the firmware did the white balance, auto gain, and in some cases how the bad pixels in the CCD were filled in along with the location of the band pixels in the image. That's pretty brutal from a finger printing point of view and bad pixels location more or less uniquely identifies the camera. Even for cameras where one can't find the bad pixel location, the issues of what values the camera tried to set the auto gain and white balance for often leak a ton of information about what type of camera was used. 

Now, I'm sure we need a long conversation about the fingerprinting of all all these API at some point and that will impact what parts of the API can be used at which point. But I don't see how that has to do with the basic idea of the constraints proposal. 

> or give developers the nightmare task of working out a meaningful subset of complex constraints to target their applications at.
> User authorisation to access the camera should not come with the need to hand over my 'identity' or give sites the ability to track me across sessions.

sure - I agree that is one use case. Again, don't see how that makes constraints a bad idea. 

> Accessing the camera should not require deep internal knowledge of codecs or their parameters (not to mention ensuring I'm using the right combinations, names, units and scales). 

total agree and the proposal does not do this

> 
> As a simple thought experiment: I could be identified in a private tab despite not being logged in had I previously logged in to the service in a non private tab at any prior point based on the unique combination of constraints created by my current hardware, OS, webcam type, drivers installed and user preferences. This group just doesn't seem to have evaluated those aspects of the proposal.

Though I agree that more thought is needed on that, I think the WebRTC WG has discussed theses at many points and raised many issues. The privacy issues are huge with accessing a camera - work is proceeding with that, but, as much as I know it will pain you, me, and Anant, all of the access to microphones and cameras and codecs will make things worse to varying degrees - we will need to decide what the threat model is, and what tradeoffs we want to make. Not using constrains does not change the work that needs to be done with respect to this. 

> 
> As a secondary example: as a developer I set my web app to only work with cameras capable of providing a resolution above 1024x768 that provide 60+ frames per second. I have just excluded ~90% of users from even trying to use my web app. Why did I choose these values?

Clearly because that is what the application needed. 

> What about the developing world with inferior hardware and software that didn't even cross my mind? Do I _really_ need these constraints for my web app to work or could we just try to work with what the users have actually got?

This question is verging on lame - if the developer set them as mandatory, clearly there was a need. 

> In reality lesser values would have worked

Really ? if that was the case the developer would have set them as optional not mandatory. An application is welcome to set nothing in the constraints in which case the API will be as you suggestion. However, many developers have made it very clear that they need more control than this. 

> just fine but I didn't add them because I couldn't possibly begin to imagine where to begin with that or where to set the threshold. Instead I just wish the user agent would give me a video stream that will work everywhere or just give me the 'best stream you've got' and we'll go from there. If my e.g. Image recognition doesn't work at the point of actually trying it then I can honestly say that it wasn't because we accidentally excluded 90% of the web before they even got to try it.
> 
>> 
>>> The upshot of this is that we will not be able to claim conformance to the getUserMedia specification if this is added.
>> 
>>> This and the fact that there is a lot of hidden complexity and a lot of unresolved issues in this proposal
>> 
>> Really? more complex than what? This is the problem: many of us can not figure out what it is you want - all we know if you don't like anything that is proposed. So what is it exactly that you think this is more complex than. And what requirements do you think we need to meet - for example, most the people I have talked to think that we need to be able to indicate some hints about the desired resolution of image capture. 
> 
> This is more complex than _nothing at all at the point of media stream acquisition_ which has proved capable of handling all of our tried and tested use cases to date.

The WebRTC WG seemed pretty clear on they needed more than the nothing you have today. Obviously the chairs can fill you in on past meetings but rooms with a lot of people in them seemed pretty unanimous in this. 

> In the course of prototyping on our implementation of this over the last 15 months, not a single developer has requested the features proposed by constraints.

> Not a single use case purporting to its need has yet to stand up to much scrutiny either. _If someone has a use case for constraints *at getUserMedia acquisition* that stands up to scrutiny then it would be very helpful.

This has been discussed to death - we have a ton of use cases. Your answer to all of them is you don't accept that use case. It's not a discussion I can really figure out how to engage in.

> 
> My point is (and has been) that this is the wrong point to be asking about resolutions or frame rates or the like. Those characteristics imply a secondary usage such as recording or streaming and should be requested independently at those points. Constraints should live at those points (if anywhere). An acquired media stream will give you whatever the platform is capable of providing which you can further manipulate downstream

Lots of USB camera can not simultaneously provide full resolution and full frame rate. Only you application knows which it would prefer. If you don't reply to anything else in this email, please think about this and let me know what you would propose. 

> 
> On the contrary at the point you obtain a media stream there are some (very few) meta characteristics that you _do_ want to request but nothing to do with the current proposal. Right now we're looking for a simple way for a page to request the front or back-facing camera. That is not a mandatory request since either camera will work but in some cases while a developer may know best which camera type is best suited for their application, and a user agent will do it's best to suggest the user to authorise the requested camera, a user may want to, and should be able to, choose the other camera.

> 
>> 
>>> is likely to significantly delay delivery of the core getUserMedia specification in the standards process.
>>> 
>>> It would be preferable if trusted environment features are pursued in a separate specifcation for now and the open issues on this proposal, of which there are many, are pursued orthogonally to the main task of nailing down the core features of the getUserMedia specification.
> 
> [1] http://dev.opera.com/articles/view/getusermedia-access-camera-privacy-ui/

Look, what we are proposing here would allow you to do everything that API does with JS function calls that are seem pretty much about the same as that API plus a few more curly, round and square braces here and there. If opera chooses to reject all mandatory constraints and ignore all optional constraints, it seems like the implementation will be about the same too.

Received on Friday, 18 May 2012 14:07:52 UTC