Re: [mediacapture-main] Spec should provide algorithms to expose individual devices via enumerateDevices (#703)

Some recent notes 

To resolve a series of issues the formal algorithm should be 

1. `enumerateDevices()` => lists all devices => before current `getUserMedia()` algorithm
2. For backward compatibility `getUserMedia()` implicitly calls `enumerateDevices()` => lists all devices 
3. User can select any and all devices for that permission session, which avoids needing to execute `getUserMedia()` more than once
4. All of the captured devices each have single `MediaStreamTrack` representation of capture, necessarily meaning there can be more than 1 microphone and 1 camera in `MediaStream` from `getUserMedia()`; it is up to user to process the tracks thereafter; there should not be an asumption that only 1 microphone and 1 camera are being captured for the duration of permissions, with speakers, cameras and other accessible media capable of being added as a device during permission being granted
5. However, to reign in confusion as to which devices are being captured the list of captured devices needs to be dynamic; and to do that with existing `enumerateDevices()` would mean the function must perform at least two different task; replace `getUserMedia()` as primary permission and device selection mechanism and thereafter list captured devices. 
6. Avoid overloading algorithms designed for a different purpose
&. Describe a `getSystemDevices()` function used to first get permission to list all devices, not just microphone and camera; user selects devices for that permission session, thereafter `enumerateDevices()` lists only those initially selected devices for that session; if plug-and-play is expected to work `ondevicechange` needs to work by user checking to watch for plug-and-play that the user explicitly grants permission to do so for meaning `enumerateDevices()` will be consistent and reliable due to being dynamic, otherwise only return selected devices
7. The algorithm needs to be basic enough to be implemented universsally with the same code, to avoid retuning back to this issue due to Media Capture and Streams not mandating a uniform UI, to that end a simple form that can be compised of plain text to either check or not check spekers, monitor of device, cameras, microphones, MIDI, et al., e.g., (https://github.com/w3c/mediacapture-main/issues/640#issuecomment-653108135) to avoid workarounds and inconsistent behaviour moving forward it must be acknowledged that `getUserMedia()` is capable of and is being used for other or more than 1 micorphone and 1 camera, that is, an extensible API design-pattern from the beginning instead of attempting to amend the functionality of prior art
8. Thereafter `enumerateDevices()` must only return devices selected during the `getSystemDevices()` selection, with `ondevicechange` being the capability to discover new devices - that the user explicitly grants permission to do so; for separation of concerns

`getSystemDevices()` => `[audio output devices (speakers), cameras, monitors of audio devices (What-U-Hear), microphones, ...]`
=> User selects any and all devices they want to capture, this is global permission grant for this session, encrypt, de-fingerprint surface, et al here.
=> If `getUserMedia()` is called 1 microphone, 1 camera selected for that puprose by user in `MediaStream`
=> `enumerateDevices()` returns only devices selected by user, makes sure the device listed is the one being captured else loses value with regard to reliability https://github.com/w3c/mediacapture-main/issues/693

-- 
GitHub Notification of comment by guest271314
Please view or discuss this issue at https://github.com/w3c/mediacapture-main/issues/703#issuecomment-653822239 using your GitHub account

Received on Saturday, 4 July 2020 23:20:01 UTC