[w3ctag/design-reviews] Updated review of Web Neural Network API (Issue #1072)

anssiko created an issue (w3ctag/design-reviews#1072)

Hello TAG!

I'm requesting an updated TAG review of WebNN API - previous TAG review: [#933](https://github.com/w3ctag/design-reviews/issues/933)


  - Explainer¹: https://github.com/webmachinelearning/webnn/blob/main/explainer.md
  - Specification: https://www.w3.org/TR/webnn/
  - WPT Tests: https://github.com/web-platform-tests/wpt/tree/master/webnn
  - User research: N/A
  - Security and Privacy self-review²: https://github.com/webmachinelearning/webnn/blob/main/security-privacy.md
  - GitHub repo: https://github.com/webmachinelearning/webnn/
  - Primary contacts:
    - Anssi Kostiainen, @anssiko, Intel (chair)
    - Dominique Hazael-Massieux, @dontcallmedom, W3C (staff contact)
    - Ningxin Hu, @huningxin, Intel (editor)
    - Dwayne Robinson, @fdwr, Microsoft (editor)
  - Organization/project driving the specification: [W3C Web Machine Learning Working Group](https://www.w3.org/groups/wg/webmachinelearning/)
  - Multi-stakeholder support³: [Participation](https://www.w3.org/groups/wg/webmachinelearning/participants/) by all major browser vendors. [Implementation experience](https://webmachinelearning.github.io/webnn-status/) across major OSes, platform APIs and key JavaScript ML frameworks.
    - Chromium comments: [High-level feedback addressed](https://www.w3.org/2024/10/17-webmachinelearning-minutes.html#dc9a). Informed by this feedback, the group has initiated work on [core operator set](https://github.com/webmachinelearning/webnn/issues/573).
    - Mozilla comments: N/A, Mozilla recently joined the WG
    - WebKit comments: N/A, Apple actively participates and contributes in the WG
  - Status/issue trackers for implementations⁴: [Chrome Platform Status](https://chromestatus.com/feature/5176273954144256), [Edge WebNN Developer Preview](https://microsoft.github.io/webnn-developer-preview/)

Further details:

  - [x] I have reviewed the TAG's [Web Platform Design Principles](https://www.w3.org/TR/design-principles/)
  - Previous early design review, if any: [2000](https://github.com/w3ctag/design-reviews/issues/570), [2022](https://github.com/w3ctag/design-reviews/issues/771), [2024](https://github.com/w3ctag/design-reviews/issues/933)
  - Relevant time constraints or deadlines: We plan to publish a new CRS during Q2 2025
  - The group where the work on this specification is currently being done: [W3C Web Machine Learning Working Group](https://www.w3.org/groups/wg/webmachinelearning/)
  - Major unresolved issues with or opposition to this specification: none
  - This work is being funded by:

You should also know that...

Below is a summary of substantive changes since the [previous TAG review in 2024](https://github.com/w3ctag/design-reviews/issues/933) to assist you in this delta review.

Significant new features:

- Add 12 new operators ("Wave 3") to support well-known transformer models based on an analysis of [top 20 downloaded Hugging Face models](https://github.com/webmachinelearning/webnn/issues/375#issuecomment-2292466613). The models comprise of a diverse set of tasks, including Small Language Models and Large Language Models, object detection, sentence similarity, feature extraction, speech recognition, image segmentation, text classification, and text generation. See [issue](https://github.com/webmachinelearning/webnn/issues/375), [PR](https://github.com/webmachinelearning/webnn/pull/805), [presentation](https://lists.w3.org/Archives/Public/www-archive/2024Sep/att-0014/WebNN_Operator_Update_Wave_3.pdf) for details on these operators and motivation.
- Add [`MLTensor`](https://www.w3.org/TR/webnn/#mltensor) interface that represents a tensor, an input or output to an [`MLGraph`](https://www.w3.org/TR/webnn/#api-mlgraph). Update [`MLContext`](https://www.w3.org/TR/webnn/#mlcontext) interface with new [`createTensor()`](https://www.w3.org/TR/webnn/#api-mlcontext-createtensor), [`readTensor()`](https://www.w3.org/TR/webnn/#api-mlcontext-readtensor), [`writeTensor()`](https://www.w3.org/TR/webnn/#api-mlcontext-writetensor) methods, change [`compute()`](https://www.w3.org/TR/2024/CRD-webnn-20241128/#api-mlcontext-compute) to [`dispatch()`](https://www.w3.org/TR/webnn/#api-mlcontext-dispatch) for scheduling the workload, add new `destroy()` method for [`MLContext`](https://www.w3.org/TR/webnn/#api-mlcontext-destroy), [`MLGraph`](https://www.w3.org/TR/webnn/#api-mlgraph-destroy), [`MLTensor`](https://www.w3.org/TR/webnn/#api-mltensor-destroy). These changes allow buffer reuse, chained inference, better resource management with explicit destruction, and the opportunity to share memory with WebGPU, minimizing copies. See [MLTensor Explainer](https://github.com/webmachinelearning/webnn/blob/main/mltensor-explainer.md) for details.

Other substantial changes:

- Remove [`MLDeviceType`](https://www.w3.org/TR/2025/CRD-webnn-20250131/#enumdef-mldevicetype) abstraction based on implementation feedback, see [Device Selection Explainer](https://github.com/webmachinelearning/webnn/blob/main/device-selection-explainer.md) for details
- Expose the API to shared workers and service workers to allow shared logic in a worker and use in web extensions
- Add [`MLContext.lost`](https://www.w3.org/TR/webnn/#dom-mlcontext-lost) attribute to signal context lost error at the earliest opportunity for improved debuggability
- Add [`opSupportLimits()`](https://www.w3.org/TR/webnn/#api-mlcontext-opsupportlimits) to allow checking whether operators/types are supported before creating a graph
- Enable [`Float16Array`](https://www.w3.org/TR/webnn/#appendices-mloperanddatatype-arraybufferview-compatibility) by default, note `Uint16Array` workaround informatively
- Convert [`MLOperand`](https://www.w3.org/TR/webnn/#api-mloperand) methods into readonly attributes (`dataType`, `shape`) per [Web Platform Design Principles](https://w3ctag.github.io/design-principles/#attributes-vs-methods)
- Add optional [`MLOperatorOptions.label`](https://www.w3.org/TR/webnn/#dom-mloperatoroptions-label) for more diagnosable error messages
- Add [`MLNumber`](https://www.w3.org/TR/webnn/#api-mlnumber-typedef) for specifying numeric inputs of any type
- Add resource contention and denial of service considerations to [security considerations](https://www.w3.org/TR/webnn/#security)

The group would like to seek TAG's explicit advise on the following:

- Operator naming 2D vs 2d https://github.com/webmachinelearning/webnn/issues/821

-- 
Reply to this email directly or view it on GitHub:
https://github.com/w3ctag/design-reviews/issues/1072
You are receiving this because you are subscribed to this thread.

Message ID: <w3ctag/design-reviews/issues/1072@github.com>