Re: [ambient-light] Request for Ambient Light Sensor web developer feedback (#64)

At the 2021 Q2 DAS meeting, we discussed how to advance the ALS spec, potentially by moving the Ambient Light Sensor into getUserMedia in order to benefit from the existing privacy and UI framework that could be used to gate permissions for this data.

### ALS inside getUserMedia

This clever hack would potentially provide an expedient foundation to bring ALS into the web platform.

However, having looked into this further, I'm not sure this worth pursuing:

  * The existing `AmbientLightSensor` spec and Chrome implementation provides an API that emits data on state change, which is a different model to a `MediaStreamTrack` continuous data stream. This would impact existing use cases by requiring them to manage change state themselves and dealing with a lot of irrelevant data, especially considering the resolution of ALS is limited to 50 lux intervals.
  * An ambient light data stream is difficult to categorize in a Media Streams API context, without introducing undue deviance. If we entertain this idea further: 
       * We thought of the ALS sensor stream as a 1x1 camera stream providing data in a single dimension (lux, rather than an RGB tuple).
       * To produce a clean and progressive API surface, I'd suggest adding a new "sensor" value to [MediaStreamConstraints](https://developer.mozilla.org/en-US/docs/Web/API/MediaStreamConstraints), then implementing a `SensorTrack`.
       * Taking this approach, to do this cleanly, the rest of the API should probably be implemented. This would introduce scope creep for the use cases described so far. Alternatively, we could implement the *bare minimum*, but IMHO, doing so would introduce a strange API surface.
       * As far as a "full" `SensorTrack` API implementation is concerned, the existing use cases don't require applying constraints such as sensitivity and resolution, and one could argue that the visibility and success of constraint application could be used for fingerprinting.
  * If we simply said "to get devicelight events, use `getUserMedia({ video: true })`", this would introduce more dilemmas for all use cases not requiring a media stream:
      * displaying the camera icon wouldn't be quite right in terms of UI design - and that would introduce more scope creep
      * what would happen if we got camera permission and then closed the stream after acquiring permission?
      * how would this work with OS level settings to block cameras but not explicitly block other sensors?
  * This would conflict with and confuse the generic sensor work, which is arguably more mainstream.

In short, having looked into this, I'm not sure that `getUserMedia` would represent a fast way of making this data available for use cases: it would present a new and unique UX challenge for browser vendors, and present an unusual or quirky API surface for developers to interact with.

### Bringing this capability into the Web Platform using the specs we have today

Per the above, I would prefer to help tackle the remaining problems with generic sensors to achieve an outcome where we have a sensible and standard way to access this and other APIs.

As we know, the way that existing sensors like the gyroscope are currently accessed is generally to bind to the `devicemotion` / `deviceorientation` event on window. In Safari you'd call `DeviceMotionEvent.requestPermission()` before receiving this information, and the information is further gated by permissions policy (or feature policy 😉).

Today, web developers requiring motion data can obtain it in this way, feature detecting if further permission prompting is required, and handling that as needed. It isn't the cleanest of APIs but it's fairly uncontroversial barring a few feature policy issues which are getting fixed.

The way the existing Chrome implementation or ALS works is perfectly suitable for _my_ use case at the moment:
  * 50 lux resolution;
  * emitting on change;
  * complying with permissions policy (`ambient-light-sensor`);
  * speculatively in WebKit browsers, calling `DeviceLightEvent.requestPermission()` if required.

We also discussed that the concerns around privacy intrusion have been disproven, are generally of low severity, are mitigated by reducing resolution and frequency of readings, and may benefit from further mitigation strategies in the future through the generic sensor spec.

As an example, [this blog post](https://blog.lukaszolejnik.com/stealing-sensitive-browser-data-with-the-w3c-ambient-light-sensor-api/) shows how one could determine browser history using devicelight events and inspecting visited link styles, logos and so on, before resolution was reduced. Lukasz's post is wonderfully creative, and has implications on other areas this group is focusing on such as the Wake Lock API, but the techniques shown also rely on a lot of modern security tooling like Content Security Policy either being misconfigured or simply not present.

I would point out that [the use case I'm advancing](https://www.iproov.com/) works in a similar, but higher fidelity way to replicate these scenarios, as that's a major part of its value proposition. It fundamentally differs only in the way that it needs to produce a correlation score of colours flashed on the screen to produce a true/false pass result, rather than estimating websites visited, user behaviour, or some other sensitive credential.

In order to achieve acceptable precision, my own use case requires a full RGB camera feed. The reason why my use case requires ambient light readings is because it can only reach a high degree of confidence _without_ environmental light introducing too much noise. I can't envisage how a single lux data stream, even at high precision or frequency, could do this alone -- but would honestly be fascinated and grateful if someone out there could show me how! 😉 

### Summary and my own thoughts on next steps

To summarise, I do not believe there is much risk of harm in introducing ALS as it is today, or even under the `devicelight` event. What we have today meets my use case, and the security and privacy implications can be mitigated with the appropriate tooling (CSP and Permissions Policy).

In the spirit of moving things forward, perhaps it would make sense to keep on the look out for use cases and slightly reduce the existing spec's scope, if needed, and proceed from there?

Thanks for reading my massive wall of text!

-- 
GitHub Notification of comment by willmorgan
Please view or discuss this issue at https://github.com/w3c/ambient-light/issues/64#issuecomment-817129135 using your GitHub account


-- 
Sent via github-notify-ml as configured in https://github.com/w3c/github-notify-ml-config

Received on Saturday, 10 April 2021 12:29:28 UTC