Re: Call for Consensus (CfC): Face Detection API (PR 78 in MediaCapture-Extensions)

I object to merging the Face Detection API in PR 78 due to the following concerns:

  1.  Scope of applicability. Currently the API in PR 78 proposes to provide hw acceleration for Face Detection based on camera driver support.  Tying support for accelerated Face Detection to support in a camera driver seems unlikely to provide wide coverage, since it is likely to only be supported on new camera models.  This will be frustrating for applications that do not wish to develop their own face detection models.  Those applications that have their own face detection models, will probably choose instead to leverage general acceleration approaches supported with Web ML platforms such as Tensorflow.js, which have wider coverage (e.g. by using WebGL for acceleration).
  2.  Variance of results. Since the actual models used may vary by camera, these APIs can potentially give varying results depending on the hardware, which will impose a support burden on applications, which could need to maintain maintain a camera blacklist, which could be difficult to develop without the ability to identify the camera hardware, which could be considered a fingerprinting risk. This problem will not arise for applications utilizing an existing face detection model written for an ML platform, since those models will yield the same results, albeit with better or worse performance.
  3.  Generality.  As noted in Issue 79<https://github.com/w3c/mediacapture-extensions/issues/79>, VideoFrame metadata is a kind of Segmentation metadata, which has a number of potential uses, such as encoder optimization. I'd therefore prefer to solve the problem more generally, with face detection being one type of segmentation.


===============================================


This is a Call for Consensus (CfC) on the Face Detection API contained in PR 78 of the Mediacapture-Extensions document.

PR 78 is available for inspection here:
Add face detection constraints and VideoFrameMetadata members by ttoivone · Pull Request #78 · w3c/mediacapture-extensions (github.com)<https://github.com/w3c/mediacapture-extensions/pull/78>

A "rich diff" can be found here: Add face detection constraints and VideoFrameMetadata members by ttoivone · Pull Request #78 · w3c/mediacapture-extensions (github.com)<https://github.com/w3c/mediacapture-extensions/pull/78/files?diff=split&w=0&short_path=dc726c5#diff-dc726c5254fb7ca8b63371f61c87fdf33312f7e83a966cb05a2f8ad2c3af54ef>

The GitHub Issues list is here: Issues · w3c/mediacapture-extensions (github.com)<https://github.com/w3c/mediacapture-extensions/issues>

In response, please state one of the following:


  *   I support merging the Face Detection API in PR 78 into the MediaCapture-Extensions document.
  *   I object to merging the Face Detection API in PR 78 due to the following Open Issues <Issue number #s>

The Call for Consensus (Cfc) will last until midnight Pacific Time on January 16, 2023.

Bernard Aboba
For the Chairs