Re: [mediacapture-main] Bug in spec: circular dependency for enumerateDevices() (#709)

> This is a lot of input, thanks @hills!
> It seems there is one precise identified issue we should try to address quickly (NotReadableError case) and more longer term proposals (design thoughts and potentially new API). It might be easier to discuss the proposal/new API as a specific issue and keep discussing the NotReadableError case here.

I agree, there is a short term vs. long term proposals here. I just wasn't sure that posting 'ideas' in ticket form is good etiquette, but I am very happy to do that if preferred.

> Diving specifically about the NotReadableError case, I can see two possibilities:
> 
>     * User Agent should be able in the middle of the getUserMedia algorithm to select another matching device if the one that is selected is NotReadableError. If all devices have issues, return "NotReadableError", no need to expose device info. It is unclear to me whether this requires a spec change. At least a note clarifying this is allowed would be good.
> 
>     * In case user granted permission and device is NotReadableError, set device information exposure to true. This requires a spec change. This also weakens a bit the model, in terms of privacy.
> 
> 
> I tend to prefer option 1. It does not seem great that a web page would have to iterate through all devices if getUserMedia({ video : true }) returns ReadableError. It should really be the responsibility of the User Agent to try as much as possible to fulfill the page request.

I don't think the two are mutually exclusive?

Frankly, the moment the user grants permission, that should be it (and I believe that would 'solve' the present ticket). That action should relate to device exposure on its own. It's very strong as it is, because of explicit action by the user. I'm afraid I don't understand how waiting for the device to open successfully strengthens privacy. I think it's actually the opposite effect; the user's understanding is complicated and weakened if it's coupled to some other future action, rather than a simple 1:1 relationship with them clicking 'allow'.

Now for the other issue. Can I clarify, you would propose to ignore 'exact' deviceID constraint if the device can't be opened? Because if I say it like that it surely can't be justified!

Even if it's not an 'exact' constraint or not, device selection logic is going to perform unusually for users in some example cases:

* I have a USB mixer, but if this can't be opened the software will fallback to, basically, listening to the conversation in the room from my internal mic. That can't be good.

* There are other transient events that will prevent a device being opened. For example, a previous app has my camera open at 480p 100Hz. My device doesn't support 1080p at 100Hz, so when I getUserMedia(1080p) I get a different camera(!)

* A previous app opens the audio device with echoCancellation, but this one asks without it. If we pick a different mic, the audio often 'looks' the same to the eye, but is wrong and it's expensive to find out later.

I think it's a noble aim to feel the API is simpler if deviceID as just another constraint. But reality some of the constraints tell us something about the source itself (ie. which way the camera/mic is pointing) vs. how we wish to capture it.

And so in my long term proposal (part 2) attempts to embody this cases without suprises for users or developers.

-- 
GitHub Notification of comment by hills
Please view or discuss this issue at https://github.com/w3c/mediacapture-main/issues/709#issuecomment-672970528 using your GitHub account


-- 
Sent via github-notify-ml as configured in https://github.com/w3c/github-notify-ml-config

Received on Wednesday, 12 August 2020 16:13:15 UTC