Re: GamepadObserver (ie. MutationObserver + Gamepad)

On Sat, Aug 4, 2012 at 12:52 AM, Glenn Maynard <glenn@zewt.org> wrote:

> On Mon, May 7, 2012 at 2:33 PM, Scott Graham <scottmg@chromium.org> wrote:
>
>> - As you point out, the 360 controller is by far the most common PC
>> gamepad, and on its native platforms its standard API XInput is
>> exclusively polling based [4]. That doesn't mean we must or even
>> should follow that design, but it does mean that even if we design a
>> bunch of fancier/cleverer APIs we're still going to be largely
>> constrained by that API. That is, there's not going to be any extra
>> information in events that are synthesized from polled data. [5]
>>
>
> Not exactly, since you can poll devices natively with these kinds of APIs
> much more quickly (without being excessively wasteful) than you can in JS.
>  Polling at 120Hz (or even 1000Hz) natively, in a well-tuned thread, is
> much more realistic than doing it in JS.  You can also do it in a
> high-priority or realtime thread, so timestamps are as accurate as possible.
>
Also don't forget that the browser only needs to poll once at each poll
iteration and then the event queues can be multiplexed to any page reading
from them, vastly more efficient.


>  > - Native deadzone handling (by the OS or internally to the device) is
>> the
>> > only way to correctly handle deadzones when you don't have intimate
>> > knowledge of the hardware.  It should be explicitly (if non-normatively)
>> > noted that native deadzone handling should be used when available.
>>
>> This has been discussed before, and it's certainly important. The one
>> complexity that needs to be solved is handling 1D vs. 2D dead zones
>> for axes. That is, should the X and Y be deadzoned as independent 1D
>> axes, or should they be deadzoned as a 2D vector. I guess this could
>> either be left to non-normative discussion and hope the
>> implementations do something sensible, or additional API surface will
>> be needed for a full solution. (I would probably lean towards
>> non-normative, and ignore the separate 1D axes problem, myself.)
>>
>
> Some hardware does deadzoning internally.  Also, the type of deadzoning
> may depend on the device; the correct answer may be different for gamepad
> thumbsticks and joysticks, for example, and devices with different amounts
> of slop need dead zones tuned differently.
>
> I think this should be entirely up to the implementation.  However, it may
> be worth a note that it *is* intended that deadzoning be handled either by
> the browser or somewhere lower down the stack, and not by web developers
> (up the stack).
>
There is a snag with deadzoning and axis ranges. Driver enumeration of axes
of an auto-calibrated device (or any device really) is often not delivering
any usable information on this. There is a zoo of range/calibration
utilities out there alongside some built-in utilities (don't know if
windows still has those) which may or may not work satisfactory. I've
tested a variety of devices for these issues, and after some work I came to
the conclusion that the only way to reliably determine any dead zones and
ranges is to let the user max out the axes and remember that.


>  > I'd suggest a halfway point between polling and events: a function to
>> > retrieve a list of device changes since the last call, and an event
>> fired on
>> > the object the first time a new change is made.  For example,
>> >
>> > var gamepad = window.openGamepad(0);
>> > gamepad.addEventListener("input", function(e) {
>> >     var changes = gamepad.readChanges();
>> > }, false);
>> >
>> > with changes being an array of objects, each object describing a
>> timestamped
>> > change of state, eg:
>> >
>> > changes = [
>> >     {
>> >         button: 0,
>> >         state: 1.0,
>> >         lastState: 0.85,
>> >         timestamp: 1336102719319
>> >     }
>> > ]
>> >
>> > This allows using polling if your application is based on
>> > requestAnimationFrame, or events if you want to be told when there's
>> > something to read.  There isn't an excess of events dispatched, because
>> the
>> > event is only dispatched once per call to readChanges; if you only read
>> > changes once in a while you'll only receive the one event.  It also
>> solves
>> > all of the above problems with a most-recent-state-only interface.
>>
>> I think it's a loss to not have a full most-recent-state available.
>>
>
> This isn't mutually exclusive with a separate getCurrentState() method.
>
The most recent state is certainly important. But in some situations you
want access to the entire queue of events since you last asked for it. I
strongly agree that emptying the event queue is the right way to go. I
strongly disagree that this should be objects. Working trough thousands of
objects would be slow and would trigger the GC like mad. The events should
be filled into an array buffer where they can processed at near C speeds
and will not trigger the GC.


> When readChanges() happens, I guess we'd need to consider some sort of
>> staleness? What happens if a client only reads once every second? Or
>> one every 5 minutes?
>>
>
> Implementations should definitely be able to cap on the amount of stored
> data.
>
Usually you allow for a max cap (say 0.5s of data) and the user may choose
a smaller window (most users will be happy with a 30ms window of data). If
the queue is full the oldest events get overwritten.


> For the most part, events can simply be dropped from the beginning of the
> queue.  However, button release events should never be dropped unless
> there's another button depress event later in the buffer.  Otherwise,
> buttons would get "stuck" to the application's view.
>
"Getting stuck" isn't the only problem. You'll also have to be careful not
to run into replay issues (i.e. repeating a queued action 20x because the
user managed to press the button so often since the last queue clear). A
common way to do this is to allow for an action only once a frame, but this
should be entirely up to the developer.

As to the rotating queues, obviously that really just applies to axes, not
really to buttons as you don't need to worry about something that can't
trigger thousands of times a second.


>  > Any serious gamepad API must not ignore the realities of the most
>> common
>>
> > joystick layouts, most importantly the 360 controller (far and away the
>> most
>> > common PC gamepad today).  The API needs to give recommended mappings
>> from
>> > buttons and axes to the parts of those real-world controllers, so users
>> > don't have to configure devices manually.  This is absolutely critical;
>> > every native PC game that supports gamepads now "just works" for the 360
>> > controller, requiring no configuration.  (For years joystick APIs on PCs
>> > tried to pretend that joysticks could be boiled down to a bunch of
>> abstract
>> > axes and buttons.  The user experience that results in is abysmal.)
>>
>> In the prototype I wrote in Chromium/WebKit, I've used the 'id' field
>> as a lightweight version of trying to make the data somewhat usable in
>> the real world. For recognized gamepads (a handful of common pad-style
>> devices), the order of buttons and axes is normalized to a canonical
>> ordering [6], *and* the string 'STANDARD GAMEPAD' is included in the
>> 'id' field. This is only done when no information is lost by doing the
>> remapping (that is, no buttons or axes are ever removed).
>>
>
> However, this needs to go both ways: if it can't fill out the standard
> layout completely, it should use a different layout.  Otherwise, it doesn't
> work: games will map functions to buttons and the user won't have any way
> to push them.
>
This is one of those things that can be solved. Each device comes with a
device ID identifying this product line uniquely. An application could
request an "input profile", if such a profile is found in the profile
database, it is returned. If either such a profile is not found, or the
user is not happy with that profile, the application provides its own
mapping. Ideally the application would deal with "virtual controls" having
some meaning for the application which get mapped to real controls. There
is some point of inflection where both framework and flexibility is
required, as sometimes configuring a set of controls cannot be squeezed
into a standard dialog to do this.