Re: [w3c/gamepad] Make gamepads accessible by web worker (#37)

kettle11 left a comment (w3c/gamepad#37)

I wanted to also shared my experience trying to make this work for Rust / Wasm game engines.

Game engines usually follow a basic loop [process user input] -> [update simulation based on user input and passing of time] -> [render frame] -> [repeat].

OffscreenCanvas is ideal to use for various reasons. Avoiding main thread stutters is one of them, but a secondary reason is it allows Wasm projects to share almost entirely unmodified threading logic with native platforms. As an example it's common for game engines to dispatch a bunch of subtasks just before rendering to worker threads, to parallelize work, and then block waiting for the tasks to complete. On the main thread blocking is forbidden, so moving the engine's simulation / rendering thread to a WebWorker is advantageous if you want to share parallelization logic mostly unmodified with native.

With that motivation for using worker threads established the issue becomes getting user input from the browser's main thread to the engine's simulation / rendering thread.

For mouse and keyboard events there's a simple approach: as events come in push them to a queue in Wasm memory that the engine's thread reads from just before it simulates / renders a frame. That message passing is low latency, but of course if the main thread has any lag those events can be delayed.

For Gamepad APIs the situation is more tricky because the API is polling based. Ideally when a frame is simulated by the engine thread the Gamepad Input should be as fresh as possible. Frames are kicked off by requestAnimationFrame and have a tight window to simulate the frame and render it.

On a native platform the engine's simulation / rendering thread would poll the gamepad right at the start of the frame. On web what is analogous? Well one way is to 'wake' the main thread, have it poll the gamepad, and have the simulation / rendering thread **block** waiting for the result. The mechanism to wake the main thread can either use postMessage or asyncAwait, but either introduces some latency and during that roundtrip (wake browser's main thread -> poll gamepad -> send state back) the browser's simulation / rendering thread will be blocked.

In an ideal scenario, with a browser main thread with nothing happening on it, on my computer this results in ~0.13ms to ~0.25ms of blocking the engine's simulation / rendering thread. This isn't terrible! At 60 fps that eats up ~1.5% of the 16ms window available for simulation + rendering. Of course that time is subject to variation, depending on how busy the main thread is. If the main thread hitches for too long the entire game's visuals will stutter.

An alternative is to have the main thread rapidly poll the gamepad API and continuously send those events to the game engine's thread which will read whatever input happens to be available. Using `setInterval` this could poll the GamePad API every ~4ms on the main thread. In this case the engine's thread wouldn't block, so there'd be no wasted frame time, but the 'staleness' of Gamepad events could be up to ~4ms in the worst case. That itself isn't terrible, but there are other sources of input latency that accumulate. If the main thread hitches for too long it would feel like the controller has disconnected momentarily.

That's longwinded, but the point is that the status quo is workable but not ideal. Tradeoffs between latency, simulation time, and user experience need to be made and the solutions aren't exactly elegant.

-- 
Reply to this email directly or view it on GitHub:
https://github.com/w3c/gamepad/issues/37#issuecomment-2741243331
You are receiving this because you are subscribed to this thread.

Message ID: <w3c/gamepad/issues/37/2741243331@github.com>