W3C home > Mailing lists > Public > public-media-wg@w3.org > July 2021

Chairs' decision: Should WebCodecs be exposed in Window environments?

From: Chris Needham <chris.needham@bbc.co.uk>
Date: Thu, 29 Jul 2021 16:11:41 +0000
To: "public-media-wg@w3.org" <public-media-wg@w3.org>
Message-ID: <AM0PR01MB5201A05344B40F4F0EAF56E1A8EB9@AM0PR01MB5201.eurprd01.prod.exchangelabs.com>
Dear all,

In the call for consensus on the question of whether to expose WebCodecs in Window context [1], six replies said that they were in favour of Window exposure at this time, and two said to defer. In addition, we take into account feedback received [2] from application developers who are not W3C members, where a number of people indicated support for Window exposure, and one was initially against but has since revised to be in support.

Overall, we see that there's a consensus between Google and application developers, but not between all major browser implementers, and there is not a consensus among the WebCodecs spec editors. There is a TAG consensus [3] (although not unanimous) to allow Window exposure.

As Working Group chairs our conclusion, based on the extensive discussion in the Working Group, and developer feedback, is that the strength of opposition to keeping WebCodecs as Worker-only is greater than the strength of opposition to exposing WebCodecs in Window, and so we should allow Window exposure at this time. We also see that a decision to defer would in effect become a decision never to expose in Window.

# Availability of related APIs

The availability of related APIs in DedicatedWorker context, such as Web Audio and WebRTC DataChannel, is clearly a limitation that will impact the performance and complexity of applications that use WebCodecs, because of the need to transfer data to the main thread. We note that support for these APIs is planned: Web Audio [4], WebRTC RTCDataChannel [5], but cross-browser implementation may be some time away.

This is a strong argument for allowing WebCodec exposure in Window at this time. This argument will grow less strong as more Web APIs are made available on DedicatedWorker contexts, and deferring the decision to expose WebCodec APIs to Window may allow other groups and specifications to include DedicatedWorker exposure.

# Difficulties using Worker

We recognize the difficulties developers have in using Workers. This difficulty, however, is not a major factor in deciding whether or not to expose WebCodecs in Window. Improvements, for example, to module imports and developer tooling are outside the scope of our Working Group, but should be progressed to improve usability of the platform overall.

# Application performance

There is a consensus and we agree that media processing in general should happen in a Worker context. Not all use cases require this, though, e.g., non-realtime transcoding. Use of Workers also adds overhead in terms of increased memory and CPU usage and message passing latency. Worker-only exposure would be a constraint that disadvantages devices with limited capabilities such as TVs.

It's true that doing too much work on the main thread leads to jank and reduced UI responsiveness. This applies in general, so is not an issue specific to WebCodecs. There are any number of ways in which web developers can build unresponsive websites, and so we do not think this is sufficient reason not to allow WebCodecs in Window context.

Web developers will need to be aware of the performance implications of using WebCodecs, but we think that performance issues will be understood by the kinds of developers interested in using WebCodecs. And today, web developers in general are more aware of how to manage application performance.

To encourage good practice, we strongly recommend that all code examples in the WebCodecs specification supporting code samples in GitHub show use of DedicatedWorker, and for developer articles (e.g, on https://web.dev) and MDN documentation to do similar, and explain the reasons for this.

# Relationship to MediaStreamTrack Insertable Media Processing using Streams

It was brought up at the July 13th Media WG meeting [6] that there is a separate but parallel discussion ongoing around the proposed MediaStreamTrack Insertable Media Processing using Streams API [7]. See the Chrome Intent to Ship for this feature [8] for details.

A decision to expose WebCodecs in Window context should not be interpreted as precedent for allowing the proposed MediaStreamTrackProcessor in Window. That proposal should be discussed and considered on its own merits.

Chris & Jer (Media WG co-chairs)

[1] https://lists.w3.org/Archives/Public/public-media-wg/2021Jun/0004.html
[2] https://github.com/w3c/webcodecs/issues/211#issuecomment-848762741
[3] https://github.com/w3c/webcodecs/issues/211#issuecomment-888811018
[4] https://github.com/WebAudio/web-audio-api-v2/issues/16
[5] https://w3c.github.io/webrtc-extensions/#rtcdatachannel-extensions
[6] https://lists.w3.org/Archives/Public/public-media-wg/2021Jul/0008.html
[7] https://w3c.github.io/mediacapture-transform/
[8] https://groups.google.com/a/chromium.org/g/blink-dev/c/oo6MQoRbDXk/m/7Kjjfe9NAwAJ
Received on Thursday, 29 July 2021 16:12:39 UTC

This archive was generated by hypermail 2.4.0 : Thursday, 29 July 2021 16:12:43 UTC