Re: GamepadObserver (ie. MutationObserver + Gamepad)

On Thu, May 3, 2012 at 9:14 PM, Glenn Maynard <glenn@zewt.org> wrote:
> Here are some piecemeal thoughts on the subject of gamepads and the
> "gamepad" spec.  I havn't closely followed earlier discussions (and there
> don't seem to have been any in a while), so much of this may have been
> covered before.
>

Hi Glenn,

Thanks for thinking about gamepads and the current draft, it's much
appreciated. There's a lot of separate points here that it'd be great
to get other people's thoughts on.

A few high-level things to note in addition to my inline comments:

- As you point out, the 360 controller is by far the most common PC
gamepad, and on its native platforms its standard API XInput is
exclusively polling based [4]. That doesn't mean we must or even
should follow that design, but it does mean that even if we design a
bunch of fancier/cleverer APIs we're still going to be largely
constrained by that API. That is, there's not going to be any extra
information in events that are synthesized from polled data. [5]

- Some of the tradeoffs that you mention in terms of making the API
more usable, or adding more functionality have been discussed before.
In general the preference has been to expose data in a more
raw/unprocessed format to allow for more flexibility to the consumer
of the data. I'm not sure that the correct balance has been struck yet
though. There's a tension between trying to make something usable
out-of-the-box and avoiding getting into specifying things in too much
detail in the spec, which will be harder to evolve and update,
especially if we over-specify in a first version.


> - It should be possible to tell what's changed, not just the current state
> of the device.  Needing to compare each piece of input to the previous state
> is cumbersome.

This is the sort of thing that might go in the content helper library.
It seems reasonable in an event-based api to provide previous and
current though.

>
> - Native deadzone handling (by the OS or internally to the device) is the
> only way to correctly handle deadzones when you don't have intimate
> knowledge of the hardware.  It should be explicitly (if non-normatively)
> noted that native deadzone handling should be used when available.

This has been discussed before, and it's certainly important. The one
complexity that needs to be solved is handling 1D vs. 2D dead zones
for axes. That is, should the X and Y be deadzoned as independent 1D
axes, or should they be deadzoned as a 2D vector. I guess this could
either be left to non-normative discussion and hope the
implementations do something sensible, or additional API surface will
be needed for a full solution. (I would probably lean towards
non-normative, and ignore the separate 1D axes problem, myself.)

>
> - It's very important that it's possible to receive gamepad data without
> polling.  We don't need more web pages running setTimeout(0) loops as fast
> as browsers will let them, which is what it encourages.  Not all pages are
> based on a requestAnimationFrame loop.

Noted.

>
> - An API that can only return the current state loses button presses if
> they're released too quickly.  It's common to press a button while the UI
> thread is held up for one reason or another--and you also can't assume that
> users can't press and release a button in under 16ms (they can).
>
> - APIs like that also lose the *order* of button presses, when they're
> pressed too quickly.  (I've encountered this problem in my own experience;
> it's definitely possible--it's not even particularly hard--for a user to
> press two buttons in a specific order in under 16ms.)

Both noted, balanced against my comment about XInput above.

>
> I'd suggest a halfway point between polling and events: a function to
> retrieve a list of device changes since the last call, and an event fired on
> the object the first time a new change is made.  For example,
>
> var gamepad = window.openGamepad(0);
> gamepad.addEventListener("input", function(e) {
>     var changes = gamepad.readChanges();
> }, false);
>
> with changes being an array of objects, each object describing a timestamped
> change of state, eg:
>
> changes = [
>     {
>         button: 0,
>         state: 1.0,
>         lastState: 0.85,
>         timestamp: 1336102719319
>     }
> ]
>
> This allows using polling if your application is based on
> requestAnimationFrame, or events if you want to be told when there's
> something to read.  There isn't an excess of events dispatched, because the
> event is only dispatched once per call to readChanges; if you only read
> changes once in a while you'll only receive the one event.  It also solves
> all of the above problems with a most-recent-state-only interface.

I think it's a loss to not have a full most-recent-state available.
Many common styles of applications (rAF+games) would have to add a
bunch of boilerplate that then merges these events to maintain the
current state (especially awkward on XInput since the browser will be
splitting them out of most-recent-state!). I don't see a lot of
benefit to splitting each individual axis/button change out into a
separate object. Is there a reason you prefer that way?

When readChanges() happens, I guess we'd need to consider some sort of
staleness? What happens if a client only reads once every second? Or
one every 5 minutes?

>
> The timestamp should be as accurate as possible to when the event actually
> happened, in the clock used by Date.now(), so you can tell how long ago the
> event happened.  Accurate timestamps are important for games that require
> fine-grained timing; 2D fighters and rhythm games can make use of this, for
> example (they may render at 60 FPS but perform game logic at a much higher
> rate).

Definitely it should be as accurate as possible. I was assuming it
would be the same as window.performance.now() [1]?

>
> Finally, some considerations other than the basic API:
>
> Any serious gamepad API must not ignore the realities of the most common
> joystick layouts, most importantly the 360 controller (far and away the most
> common PC gamepad today).  The API needs to give recommended mappings from
> buttons and axes to the parts of those real-world controllers, so users
> don't have to configure devices manually.  This is absolutely critical;
> every native PC game that supports gamepads now "just works" for the 360
> controller, requiring no configuration.  (For years joystick APIs on PCs
> tried to pretend that joysticks could be boiled down to a bunch of abstract
> axes and buttons.  The user experience that results in is abysmal.)

In the prototype I wrote in Chromium/WebKit, I've used the 'id' field
as a lightweight version of trying to make the data somewhat usable in
the real world. For recognized gamepads (a handful of common pad-style
devices), the order of buttons and axes is normalized to a canonical
ordering [6], *and* the string 'STANDARD GAMEPAD' is included in the
'id' field. This is only done when no information is lost by doing the
remapping (that is, no buttons or axes are ever removed).

Adding a few "styles" of layouts (or maybe only just that one for now)
to the spec seems reasonable to me. This could just be the ordering as
defined above, along with a diagram to describe the rough physical
location of the button/axis. We could add a more formal way of
'tagging' the style too if necessary.

On the other hand, we don't to exclude the ability to use other
devices. I have had people contact me about using 3D mice [2] and foot
pedals [3] via this API, and it would seem odd not support those
similar uses. So, having the raw axis+button arrays available for more
custom devices and applications seems important too.

>
> If the gamepad model is known, it should also be explicitly exposed, so
> applications can show help texts which match the actual buttons on the
> gamepad (with a registry somewhere--which could just be a wiki
> page--describing the string to product mappings).  Every native PC game also
> now does this, finally catching up to what console games have done for
> decades.  If a web API for gamepads doesn't do this, it'll be years behind
> everything else.

This was the intention of the 'id' field. The combination of 'STANDARD
GAMEPAD' and including the USB vendor and product id, as well as other
driver name information gets us some of the way there. It's currently
ad-hoc though, and we probably want to clarify this as we understand
better how authors want to use it.

>
> Additionally, a more involved system of input profiles would be needed for
> practical forward-compatibility, eg. so applications can say "this is a list
> of device types I understand" and UAs can expose the device as the nearest
> match.  This allows old applications to work better with devices they don't
> know about, without requiring the user to manually bind buttons and axes.  I
> can't stress how important it is to not require the user to configure every
> game manually; it's simply no longer acceptable.

This gets very complex to specify, I think. We had previously pushed
this towards a content library. The standard style/layout would help
alleviate this somewhat though.

scott

[1] http://www.w3.org/TR/hr-time/
[2] http://www.3dconnexion.com/products/spaceexplorer.html
[3] http://www.sylvansoftware.com/Infinity%20Foot%20Pedals.htm
[4] http://msdn.microsoft.com/en-us/library/windows/desktop/ee417014(v=vs.85).aspx
[5] My understanding of USB is limited to a few quick searches, but as
I read it, the default USB hardware polling rate is only 125Hz = 8ms
anyway. Anyone familiar with hardware know if that applies to
many/most gamepads?
[6] http://code.google.com/searchframe#OAMlx_jo-ck/src/content/browser/gamepad/gamepad_standard_mappings.h

Received on Monday, 7 May 2012 19:33:57 UTC