Re: [w3ctag/design-reviews] Early TAG review request for Playout Statistics API for WebAudio (Issue #939)

Hi @matatk! The RenderCapacity API (#843) and the Playout Statistics API serve different information and for different purposes. As you say, the RenderCapacity API is about seeing the trend of CPU usage and adapting the playout graph as needed. It also exposes one kind of audio glitch through [underrunRatio](https://webaudio.github.io/web-audio-api/#dom-audiorendercapacityevent-underrunratio), which happens when the WebAudio graph is too complex for the CPU to deliver audio on time.

The Playout Statistics API serves information about the end-user audio experience, which includes glitches from the whole audio pipeline and end-to-end latency. These general audio glitches can happen for many different reasons including:
* High cpu usage to the point where it affects realtime threads
* Platform audio layer/driver glitches
* Web applications failing to deliver audio on time (due to slow WebAudio graphs, for example)

This means that the Playout Statistics API gives much less information about CPU usage than the RenderCapacity API, because RenderCapacity lets us know about gradual levels of CPU usage, while the Playout Statistics API only lets us know if there are actual audio glitches, one possible reason for which is critical CPU usage. This critical CPU usage is already detectable in other ways, such as delayed audio callbacks.

If we combined the APIs, then the Playout Statistics API would also have to give RenderCapacity information. If it turns out that the RenderCapacity API is too invasive, or that we want to guard the RenderCapacity API behind some additional permission, then it's better that the APIs are separate.

To clarify about the side channel: The preexisting side channel we mentioned only allows cooperating sites to communicate with each other with low precision and high latency, and only if they are open at the same time on the same device. The idea is that if one site creates critical amounts of cpu pressure on realtime threads, this can be observed by other sites in a variety of ways, including delayed audio callbacks and audio glitches. This method does not, however, allow anyone to gain information about other processes that are not actively cooperating. CPU overload can be detected using glitches, but the glitches could also mean other things like platform layer glitches or that the WebAudio graph is slow. We therefore believe that this doesn't create any real user risk, and we believe that we would not add any new risk with the introduction of this API.

We have not talked to other implementers about this proposal, we would like to pass early TAG review first.

-- 
Reply to this email directly or view it on GitHub:
https://github.com/w3ctag/design-reviews/issues/939#issuecomment-2112744241
You are receiving this because you are subscribed to this thread.

Message ID: <w3ctag/design-reviews/issues/939/2112744241@github.com>