Re: [gamepad] Allow standard-mapped devices to have extra buttons or axes & allow multiple mappings per device

Whew... trying to process all of Katelyn's email. Forgive me if I overlook
some things, but I'm going to try and hit a few points that have been on my
mind:

On Wed Oct 15 2014 at 12:43:14 AM Katelyn Gadd <kg@luminance.org> wrote:
>
> If there is one standard mapping, and it covers most users, this means
> the natural path for developers is to only support the standard
> mapping. They won't add keybinding/button remapping, they won't add
> support for multiple layouts, they might not even add support for
> choosing which of your gamepads you want to use. This is not an
> unrealistic fear: It is the status quo in virtually all console games
> and in many PC games. In some cases you can rebind by editing .ini
> files, but that is out of the question for web games since their
> configuration typically lies in not-user-editable JS on the server or
> in blobs stored somewhere (localStorage, indexed db, cookies) that are
> not easily user-editable. This 'developers only support the easiest
> thing' future is relatively likely and is actually worse than the
> previous status quo of raw-controllers-only.
>

To be brutally honest I wish that even _this_ was the status quo. In my
experience thus far:

 * Very few games on the web bother with the gamepad API, which I think
boils down to a marketing problem. I've actually had game devs say things
like "So, do you think the web will ever have a way to interact with
gamepads/joysticks/wheels/etc?" If nobody knows the API exists it's hard to
complain much about it's deficiencies. Unfortunately I'm not sure how we
solve this, but assuming that we do...

* Many of the games that I DO see utilizing the API seem to take the
approach of "Well, the dev had a KnockoffBox720Pad, so everything is hard
coded to whatever that reported." It would be great if devs even bothered
looking at the mapping field, but I don't see too many cases where they do.
Again, this feels like a marketing problem. :P


> I think the solution to this is simple: Ensure that there is more than
> one standard mapping. Find a way to make sure that developers have to
> do at least a basic amount of effort to handle things correctly. If
> the simple hack isn't sufficient for wider audiences, they will have
> an incentive to do a good job. This doesn't have to be a punitive
> measure - there are diverse controllers out there that people might
> want to use, from standard gamepads to racing wheels to flight sticks.
> Something as simple as supporting the additional inputs of the
> dualshock 3 & 4 is a good rationale here (they both have gyros &
> accelerometers; the 4 has a touchpad and touchpanel button.)
>

I'm not against adding a new mapping type. "flightstick" seems like the
obvious choice there, though the button layouts on those can vary pretty
radically. At the very least you could standardize axes layout, primary
fire, secondary fire, and hat switch layout and allow for whatever button
order seems natural after that. (Slight tangent: Does anyone know if
the HOTAS-style
controllers <http://www.thrustmaster.com/products/hotas-warthog> report
both the throttle and stick as a single device?)

I do agree that having more than one standard mapping encourages more
robust usage, assuming we overcome the previously mentioned problems of API
visibility.


> we have the *miserable* constraint where you have to push a button to
> make your controller show up. This changes the ordinal identifiers of
> gamepads from session to session, and - even worse - means that a
> controller cannot be activated by analog stick inputs, which is a real
> problem for games that start with a menu (like two of my major ports).
>

I think we could actually use axes as activation inputs with a little more
processing. The big reason for not doing it right now is that on a lot of
devices inputs that report as an axis (triggers on XInput) have a resting
value other than 0. This may also be a problem if a gamepad has been set
down in such a way that a stick is offcenter (though to be fair the same
problem exists for buttons. I have a gamepad with a stuck "X" button that
immediately activates whenever I go to a gamepad-enabled site.)

My proposed solution would be to not report an input's value unless it has
reported 0 or at least below a deadzone (post-user-agent-mapping) at least
once. This would mean that a joystick axes wouldn't report unless they had
been in their resting state at least once since the page started reporting,
and a stuck button would be a dead input instead of a continuously firing
one. The benefit is that we can then treat any non-zero inputs after that
point as intentional, including axes movement. Thus picking up a controller
and pushing the stick up/down to navigate a menu would properly activate
the gamepad API. It also wouldn't require any API change, which is
generally preferable in any case where the API has been out in the wild.

This defies user expectations and is just plain confusing. Polling an
> actual device to get its state is perfectly reasonable, but polling
> the overall state of your input devices is not so. Device availability
> is something that should be event-based (it is at a driver level
> anyway) and it should be exposed to content as events.
>

We do have device connected/disconnected events in the spec now, though
they're still subject to the user-gesture-to-activate requirements.


> The browser makes a 'best effort' at remapping, but there is no
> real facility for the content to figure out how good the 'best effort'
> was, or to opt into getting the raw state so they can do a better job.
> The fact that a 'best effort' will not always be possible means that a
> bunch of the work here - arguably some of the most important work - is
> shoved off into content, but the common case is easy enough that
> content may not cover the less common cases. This really sucks.
>

Agreed. (I think Florian and I talked about this at one point but I never
had the time to move on it.) I think it would be great to allow users to
request that they want unmapped data, at which point every device would
report raw input values. Implementation of this gets tricky internally,
since (in Chrome) all pages poll gamepad data from a single shared memory
source, but that's not a good reason to avoid doing it. That way any JS
libraries that want full control of the mapping can have it. Then we don't
have to muck around with the messy concept of "blessed" libraries.


> I think a relatively ideal - not perfect, realistic - solution in
> content can also solve problems that the browser can't solve by
> itself. In detail:
> * The blessed polyfill can be officially hosted on a
> community-controlled domain - perhaps shepherded by a known good actor
> like Mozilla or Google - that acts as a canonical location for the
> gamepad mapping support.
>

Yup, I've been arguing for this approach for a while now. I'd love to see a
community-driven mapping effort, but nobody I've talked to yet has had the
time/motivation to drive such an effort forward. I've also thought that a
browser extension that allows users to change the mappings invisibly to the
web page would be really nice, either as part of that same effort or
independently.


> Full disclosure: I was involved in the initial prototyping & spec work
> for the gamepad API that happened at Mozilla, up until shortly before
> Google's shipped prototype API became the de-facto standard. I'm
> slightly bitter, if only because the request-oriented approach to
> acquiring gamepads is what I proposed originally and fought hard to
> make happen on our side. I'm a bit sad that the current approach
> shipped because while it solves important problems, it sets us up for
> some pain in the future.
>

I didn't have much to do with the original incarnation of the gamepad API,
but I'm one of the owners on the Google side now so I'm more than happy to
try and make it better, because it unquestionably does have it's
shortcomings. Thanks for typing this up, and sorry that I haven't had a
chance to respond to each point individually! I'd love to discuss it
further, with my biggest concern being that some of it may simply come down
to finding contributors with enough time to tackle the community-driven
parts.

--Brandon