[mediacapture-main] Racy devicechange event design has poor interoperability (#972)

jan-ivar has just created a new issue for https://github.com/w3c/mediacapture-main:

== Racy devicechange event design has poor interoperability ==
The [devicechange](https://w3c.github.io/mediacapture-main/getusermedia.html#event-mediadevices-devicechange) event follows [ยง 7.7. Use plain Events for state](https://w3ctag.github.io/design-principles/#state-and-subclassing), but the _"state information in the [target](https://dom.spec.whatwg.org/#dom-event-target) object."_ is not available synchronously, so developers have to the following:
```js
navigator.mediaDevices.ondevicechange = async () => {
  const devices = await navigator.mediaDevices.enumerateDevices();
  // examine devices and compare against oldDevices to detect changes once available
  oldDevices = devices;
}
```
But this is racy. By the time you're finally examining the devices, 100+ milliseconds may have passed. A lot may have happened during that time, including more `devicechange` events, which the spec allows.

Theoretical? No. I've instrumented [this fiddle](https://jsfiddle.net/jib1/x94rLgvq/21/) from https://github.com/w3c/mediacapture-main/issues/966#issuecomment-1734133569, where I keep AirPods in their case initially, then put them on, in macOS Ventura.

Firefox is fine (baseline):
```
ENUMERATEDEVICES 1 BEGIN...
...ENUMERATEDEVICES 1 END.
ENUMERATEDEVICES 2 BEGIN...
...ENUMERATEDEVICES 2 END.
DEVICECHANGE 1 BEGIN...
ENUMERATEDEVICES 3 BEGIN...
...ENUMERATEDEVICES 3 END.
1: Switching to inserted AirPods
...DEVICECHANGE 1 END.
```
The first two are me calling enumerateDevices before and after getUserMedia, like you're supposed to, to learn of all devices.

But here's Chrome:
```
ENUMERATEDEVICES 1 BEGIN...
...ENUMERATEDEVICES 1 END.
ENUMERATEDEVICES 2 BEGIN...
...ENUMERATEDEVICES 2 END.
DEVICECHANGE 1 BEGIN...
ENUMERATEDEVICES 3 BEGIN...
DEVICECHANGE 2 BEGIN...
ENUMERATEDEVICES 4 BEGIN...
DEVICECHANGE 3 BEGIN...
ENUMERATEDEVICES 5 BEGIN...
DEVICECHANGE 4 BEGIN...
ENUMERATEDEVICES 6 BEGIN...
...ENUMERATEDEVICES 3 END.
1: ignoring duplicate event
...ENUMERATEDEVICES 4 END.
2: ignoring duplicate event
...ENUMERATEDEVICES 5 END.
3: ignoring duplicate event
...ENUMERATEDEVICES 6 END.
4: Default - MacBook Pro Microphone (Built-in) removed; reverting to Default - MacBook Pro Microphone (Built-in)
...DEVICECHANGE 4 END.
DEVICECHANGE 5 BEGIN...
ENUMERATEDEVICES 7 BEGIN...
DEVICECHANGE 6 BEGIN...
ENUMERATEDEVICES 8 BEGIN...
...ENUMERATEDEVICES 7 END.
5: ignoring duplicate event
...ENUMERATEDEVICES 8 END.
6: Switching to inserted AirPods
...DEVICECHANGE 6 END.
DEVICECHANGE 7 BEGIN...
ENUMERATEDEVICES 9 BEGIN...
...ENUMERATEDEVICES 9 END.
7: AirPods removed; reverting to Default - AirPods
...DEVICECHANGE 7 END.
DEVICECHANGE 8 BEGIN...
ENUMERATEDEVICES 10 BEGIN...
...ENUMERATEDEVICES 10 END.
8: Default - MacBook Pro Microphone (Built-in) removed; reverting to Default - AirPods
...DEVICECHANGE 8 END.
```
8 events are fired, and they're fired on top of each other, while the application is still processing the previous event(s) (awaiting enumerateDevices), causing interleaved code execution.

This would look even worse if I didn't write special guards against duplicate events, but this is still a nightmare to reason about.

This might look like an implementation bug on Chrome, except here's Safari:
```
ENUMERATEDEVICES 1 BEGIN...
...ENUMERATEDEVICES 1 END.
DEVICECHANGE 1 BEGIN...
ENUMERATEDEVICES 2 BEGIN...
ENUMERATEDEVICES 3 BEGIN...
...ENUMERATEDEVICES 2 END.
...ENUMERATEDEVICES 3 END.
1: Switching to inserted MacBook Pro Microphone
...DEVICECHANGE 1 END.
```
...and so far I haven't even inserted my AirPods yet! This is #966 where Safari fires a `devicechange` event as part of getUserMedia, tricking my fiddle into thinking a device has been inserted.

As mentioned previously, my fiddle calls enumerateDevices right after getUserMedia like you're supposed to (and have to in other browsers). But here Safari fires `devicechange` on me, causing my app to again interleave two calls to enumerateDevices, making it hard to reason about. It may even expose it to the race in the OP, since AFAIK there's no spec guarantee that enumerateDevices calls resolve in order.

Safari continued:
```
DEVICECHANGE 2 BEGIN...
ENUMERATEDEVICES 4 BEGIN...
...ENUMERATEDEVICES 4 END.
2: MacBook Pro Microphone removed; reverting to MacBook Pro Microphone
...DEVICECHANGE 2 END.
DEVICECHANGE 3 BEGIN...
ENUMERATEDEVICES 5 BEGIN...
...ENUMERATEDEVICES 5 END.
3: Switching to inserted AirPods
DEVICECHANGE 4 BEGIN...
ENUMERATEDEVICES 6 BEGIN...
...ENUMERATEDEVICES 6 END.
4: MacBook Pro Microphone removed; reverting to AirPods
...DEVICECHANGE 3 END.
...DEVICECHANGE 4 END.
```
...the rest here is fairly decent, in that calls are not getting interleaved. But like Chrome there's some funny business going on with the default device. What's that about?

This seems like an interoperability issue. For example: Device switching in Google Meet works in Chrome today but not in Firefox, which I think shows it is possible to write app code that passes a testing matrix in one implementation, yet is not interoperable, unless other browsers implement the same side effects.

This seems like a spec issue. I propose we delay `devicechange` events to not fire while calls to `enumerateDevices()` are outstanding.

I'll file a separate issue on the non-standard "Default" device that Chrome and Safari add for microphone only.

Please view or discuss this issue at https://github.com/w3c/mediacapture-main/issues/972 using your GitHub account


-- 
Sent via github-notify-ml as configured in https://github.com/w3c/github-notify-ml-config

Received on Tuesday, 26 September 2023 21:50:13 UTC