Proposal for MediaTrackCapabilities discovery via enumerateDevices()

Hello Media Capture,

This is the second proposal from the Audio WG relating to our earlier Last
Call requests. It relates to device capabilities discovery. (A third,
future proposal will address a set of related issues with the Audio Output
API, but we will skip those questions for now.)

Summary:

The proposed change will allow the MediaDeviceInfos in the array returned
by enumerateDevices()  to include a new *capabilities* attribute of type
MediaTrackCapabilities, available *only* in the already-defined case where
anti-fingerprinting filtering is not applied.

Rationale:

It is not possible for audio or video applications today to offer the user
a choice of available devices that takes account of their capabilities or
characteristics, e.g. channel count, pixel dimensions, sample rate, etc.

While it is possible to enumerate devices and obtain their names, kinds and
opaque IDs, applications also need to  a) filter device lists to only
include usage-appropriate entries (e.g. only audio inputs >= 44k sample
rate), or b) display useful information to accompany each device's name
(e.g. whether it's mono, stereo, etc.). Other use cases also exist.

The existing Media Capture API proposal already addresses potential
fingerprinting exploits by prohibiting the return of device labels unless
the user has explicitly granted access to at least one device. This
proposal would apply the same restriction to the return of device
capabilities.

Proposed changes:

- Add the following attribute to MediaDeviceInfo:

    readonly attribute MediaTrackCapabilities capabilities;

Its definition is as follows:

  * If access to this information is disallowed to the caller, the
attribute's value is null.
  * Otherwise, the attribute contains a MediaTrackCapabilities object
identical to that which would be obtained by the following algorithm:
      - Let info be the MediaDeviceInfo whose capabilities attribute is
being determined.
      - If info.type is "audioinput" or "videoinput",
        - Call getUserMedia({deviceId: info.deviceId})
        - If the promise succeeds
          - Let track be the first MediaStreamTrack in the returned
MediaStream whose type is "audio" (if the device type was "audioinput") or
"video" (if the device type was "videoinput").
          - Assign the attribute's value from track.getCapabilities()
      - Otherwise if info.type is "audiooutput" the attribute's value is a
MediaTrackCapabilities describing the capabilities of the audio output
device (if these can be obtained) or else null.

- Change the description of enumerateDevices() in 9.2.2 as follows:

In step 4, to the text " let filteredList be a copy of resultList, and all
its elements, where the label member is the empty string", append "and the
capabilities member is null".

- Change the text of section 9.2.3 (Access control model) as follows:

Modify the text "If access has been granted for a media device, the
MediaDeviceInfo dictionary will contain the deviceId, kind, label, and
groupId" to read "If access has been granted for a media device, the
MediaDeviceInfo dictionary will contain the deviceId, kind, label, groupId
and capabilities."

Thanks very much for your consideration.

-- 
.            .       .    .  . ...Joe

*Joe Berkovitz*
President

*Noteflight LLC*
49R Day Street / Somerville, MA 02144 / USA
phone: +1 978 314 6271
www.noteflight.com
"Your music, everywhere"