[mediacapture-main] Bug in spec: circular dependency for enumerateDevices() (#709)

hills has just created a new issue for https://github.com/w3c/mediacapture-main:

== Bug in spec: circular dependency for enumerateDevices() ==
If the default device fails to open (even with permissions) then it has now become impossible to use any other device.

This is because of this new condition in enumerateDevices (summarised in commit c15a432b, March 2020):

> if the browsing context did not capture (i.e. getUserMedia() was not called or never resolved successfully), the MediaDeviceInfo object will contain a valid value for kind but empty strings for deviceId,

Previously this read:

> if no such access has been granted [...]

Here's how this plays out in practice:

* getUserMedia({audio: true});    // no device ID is known at this stage
* user is prompted and grants permission to use the primary device
* device open failure: an exception happens and getUserMedia fails
* enumerateDevices();    // blocked from listing alternative devices
* dead end for the user; they have no way to select an alternative device

Chromium attempted to follow the new spec but [reverted the change](https://bugs.chromium.org/p/chromium/issues/detail?id=1111333).

In practice there are reasons a device may not be able to be opened, such as exclusive use by another application, or cannot fulfil some criteria, or just a fault. These may be platform or hardware dependent.

It looks like the summary in the commit is based on 9.2.2 "Device information exposure" which has been adjusted in commit e159c60, also in March.

I am not a spec author, I am afraid, and I would need time to fully understand the detailed steps described in the spec. But if I may suggest that it seems like the spec embodies a lot of policy that means existing special cases are causing new ones.

---

A proposal for what the user or developer experience should be that would make a lot of this simpler, whilst avoiding fingerprinting/probing issues:

Calls to getUserMedia that _do not specify a device ID_ (or specify "default") would be governed by a "permission to use your camera/microphone" dialogue provided by the browser:

* The browser should provide option to choose a device here
* The persistence of this permission does not need to be spec'd
  * Browser or user policy can device: eg. forever; this session
  * Or every call to getUserMedia()
  * Spec may demand some implicitly permitted operations; eg. if the device is already open by that page
* This means that most web developers can just call getUserMedia, once, and not worry about enumerateDevices

And then, independently a permissions flag (looks like [[canExposeDeviceInfo]]?):

* Browser prompts for "permission to use a range of media devices on your system"
* Governs access to anything involving a deviceID: eg.
  * enumerateDevices()
  * getUserMedia use with a deviceID
  * setSinkID
  * "new device available" event
* Pop up the permissions dialogue on the first of any of these API uses
  * Even if the page remembers a deviceID in a cookie, it must still have this permission to make an API call with it
* Browser policy or user sets how long this permission is granted for: eg. forever; this session; next 5 minutes
* This dialogue can provide access control
  * eg. "default devices only", "microphones only", or a specific allowlist
* These deviceIDs now act as a pass on getUserMedia(), so not subject to the checks above.

What my goals are in the above proposal:

* Clarity of not having permissions based on events ordering:
  * enumerateDevices() needing to happen after getUserMedia()
  * avoid 'fake' data now, complete data later
* Tackle fingerprinting issues; no deviceIDs granted without user permission
* User is clear which page can access which devices, not implicit based on history (#703)
* Retain compatibility with exisiting code
* Prevent most developers having to build device selection UIs unnecessarily:
  * Chromium presently omits device selection on getUserMedia, which means most developers need to call enumerateDevices at present to give good user experience, and do this after an early getUserMedia() to call for permissions.
* Leave policy to the browser
  * Spec can be simpler and less bug footprint
  * Policy is variable in different browsing environment; eg. mobile vs. desktop vs. pro user/producitivity app
  * Policy can evolve without breaking existing sites

It is good to remember that not all apps are standard video conferencing apps, and increasingly there are WebAudio apps for producivity will use multiple devices concurrently.

Please view or discuss this issue at https://github.com/w3c/mediacapture-main/issues/709 using your GitHub account


-- 
Sent via github-notify-ml as configured in https://github.com/w3c/github-notify-ml-config

Received on Monday, 3 August 2020 14:49:26 UTC