Re: Proposal to add USB keycodes to the current DOM3 key events

(Including the contents of entire document inline so that the info is
easier to skim/comment on.)

-Gary Kacmarcik (garykac@chromium.org


Proposal for enhancing the current DOM Level 3 key events with USB scancodes

Status: DRAFT
Authors: Gary Kacmarcik, James Weatherall


The current DOM Level 3 key event spec improves considerably upon the
"legacy" HTML4 key events that are currently used by most browsers.
The HTML4 events have always been problematic because the lack of a
detailed specification for the events led to implementation
inconsistencies on the various platform/browser combinations. The DOM
Level 3 proposal seeks to address this problem by specifying exactly
how these key events should be encoded.

However, there are still cases where the current proposal does not
provide enough information about the key event to perform desirable
tasks. In some cases, the legacy event values actually provide more
(internally) consistent information about the events.


Problems with the current DOM Level 3 key events
================================================

Problem 1: Matching keydown and keyup events
--------------------------------------------

The main problem with the current proposal is that it doesn't provide
enough information so that keyup events can be matched with their
corresponding keydown event. In this regard, it is unlike the legacy
events, where the keyCode value was usually sufficient to match the
keydown/keyup events.

For examples of why this is a problem, consider the following scenarios:

* Games and other applications which treat the keyboard as a large
collection of buttons need to be able to keep track of which keys are
currently being held down. Since there is no API to query the current
keyboard state, it is crucial that apps be able to match every keyup
event with the originating keydown event.

* Applications that provide remote access functionality often need to
keep track of the sequence of events that generated a particular
character. This is so that they can handle the case where the local
and remote systems have different keyboard layouts (since they may
require a different sequence of keys or modifiers to be injected on
the remote machine).

Appendix A (at the end of this document) contains some sample key
event sequences that demonstrate this problem. The simple case of
typing a shifted character (the '@' sign) works fine in the normal
case, but if the Shift key is released too early then the keyup will
have a different value than the original keydown. This inconsistency
makes it difficult to match the keyup with the originating keydown.

Note that since these event values are dependent on the current
keyboard layout, there is no simple mapping table between the shifted
and unshifted version of each key. The international keyboard layout
examples in Appendix A demonstrate this complication.


Problem 2: Identifying keys by their position
---------------------------------------------

While not as critical as the first problem, another common task for
some applications (typically games) is to handle key events based on
the key's position on the keyboard. For example, a game may want to
detect when the key next to the CapsLock key is pressed so it can be
associated with some player action. With a US keyboard, this would be
the key with the 'A' keycap, but on an AZERTY keyboard, this key would
have a 'Q' keycap.

A layout-independent way of identifying keys would be useful to games,
music/beatbox players, and other apps that prefer to consider the
keyboard as a set of buttons. A touch-typing app could also use
information to this target lessons for the home row or for the
left/right hand.

(Note that some implementations of HTML4 key events suffer from this
problem as well. For example, the keycodes in WebKit-based browsers
are based on Windows VKEY values, which are layout-dependent.)


Proposal
========

Our proposal to address these problems is to add a new field that
contains the USB Usage Page and Usage ID codes for keydown and keyup
events.

In this proposal, the KeyboardEvent would be extended to include:

   interface KeyboardEvent : UIEvent
   {
      ...
      readonly attribute unsigned long usbUsage;
      ...
   };

The usbUsage field is a combination of the USB Usage Page (upper
16-bits) and the USB Usage ID (lower 16-bits).  Some example usbUsage
codes are given in Appendix B, but a complete list can be found in the
USB HID Usage Tables documentation (see References).

The vast majority of these codes would come from Usage Page 0x07,
which contains Keyboard/Keypad values. However, many modern keyboards
contain specialized "media" keys which would have values defined in
other Usage Pages.

Providing this information in addition to the currently specified
fields would be address the problems described earlier:


Solving Problem 1 : Easy to match keydown with keyup

Since the USB key values are associated with the individual keys, the
value would not depend on the current state of the modifier keys.
Thus, the keydown and keyup events would have a value consistent
across all events generated by the same physical key.


Solving Problem 2 : Can identify key by position on keyboard

While the USB Usage IDs are given names based on the US keyboard
layout, the values are effectively scancodes, identifying the physical
key position on the keyboard. This makes it possible to write code
that can detect keys based on their position without needing to worry
about the current keyboard layout.



Appendix A - Key event sequence examples
========================================

Notes about these examples:
* Test machine: Windows 7 using a US keyboard.
* The legacy HTML4 keyCodes listed here are Windows VKEY codes used by
IE and WebKit-based browsers. They were captured on Chrome 19 /
Internet Explorer 9. Other browsers and platforms may have different
values
* The DOM3 events were captured on Internet Explorer 9.
* Note that not all of the key event fields are shown and textInput
events are ignored.
* The ↓ symbol following a key name is used to denote keydown events;
↑ is used to for keyup events.

Proposed USB Usage values are included for reference. The upper 2
bytes are the "Usage Page ID" (always 0x7 in these examples) and the
lower 2 bytes (the "Usage ID") contains the code for the individual
key.


Example 1: Typing "@" as shift ↓, 2 ↓, 2 ↑, shift ↑ (US Keyboard layout)
------------------------------------------------------------------------

The simple case that works. The DOM3 key field matches for the keydown
and keyup. Likewise for the legacy keyCode field.

                         |   Legacy  |         DOM3         |  USB  |
                         | key |char |  key  | char |  loc  | Usage |
   keydown      Shift       16    0    Shift   null   LEFT   0x700E1
   keydown        2         50    0      @       @      0    0x7001F
   keypress                 64   64      @       @      0
   keyup          2         50    0      @       @      0    0x7001F
   keyup        Shift       16    0    Shift   null   LEFT   0x700E1


Example 2: Typing "@" as shift ↓, 2 ↓, shift ↑, 2 ↑ (US Keyboard layout)
------------------------------------------------------------------------

Note that the DOM3 key field does not match for the keydown and keyup.
However, the legacy keyCode field does match.

                         |   Legacy  |         DOM3         |  USB  |
                         | key |char |  key  | char |  loc  | Usage |
   keydown      Shift       16    0    Shift   null   LEFT   0x700E1
   keydown        2         50    0      @       @      0    0x7001F
   keypress                 64   64      @       @      0
   keyup        Shift       16    0    Shift   null   LEFT   0x700E1
   keyup          2         50    0      2       2      0    0x7001F


Example 3: Typing " as shift ↓, 2 ↓, shift ↑, 2 ↑ (UK Keyboard layout)
----------------------------------------------------------------------

With a UK keyboard layout, the same problem occurs, but with different
DOM3 key values.

                         |   Legacy  |         DOM3         |  USB  |
                         | key |char |  key  | char |  loc  | Usage |
   keydown      Shift       16    0    Shift   null   LEFT   0x700E1
   keydown        2         50    0      "       "      0    0x7001F
   keypress                 34   34      "       "      0
   keyup        Shift       16    0    Shift   null   LEFT   0x700E1
   keyup          2         50    0      2       2      0    0x7001F


Example 4: Typing 2 as shift ↓, 2 ↓, shift ↑, 2 ↑ (French Keyboard layout)
--------------------------------------------------------------------------

Similar issue with French keyboard. The un-shifted "2" key is not always a "2".

                         |   Legacy  |         DOM3         |  USB  |
                         | key |char |  key  | char |  loc  | Usage |
   keydown      Shift       16    0    Shift   null   LEFT   0x700E1
   keydown        2         50    0      2       2      0    0x7001F
   keypress                 50   50      2       2      0
   keyup        Shift       16    0    Shift   null   LEFT   0x700E1
   keyup          2         50    0      é       é      0    0x7001F


Example 5: Pressing NumLock between keydown keyup (US Keyboard layout)
----------------------------------------------------------------------

A more extreme example, but this demonstrates that the legacy keyCode
field was not immune to these problems. In this case, neither the DOM3
key field nor the legacy keyCode field match between the keydown and
keyup events.

                         |   Legacy  |         DOM3         |  USB  |
                         | key |char |  key  | char |  loc  | Usage |
   keydown   Keypad 8/Up    38    0     Up     null  KEYPAD  0x70060
   keydown     NumLock     144    0   NumLock  null     0    0x70053
   keyup       NumLock     144    0   NumLock  null     0    0x70053
   keyup     Keypad 8/Up   104    0      8     null  KEYPAD  0x70060


Example 6: Autorepeating keys: shift ↓, 2 ↓, shift ↑, 2 ↑ (US Keyboard layout)
------------------------------------------------------------------------------

Unsurprisingly, the same issue occurs with autorepeating keys. Press
and hold shift, press and hold "2", release shift, wait and release
"2". Events marked with an asterisk (*) are repeat key event.

                         |   Legacy  |         DOM3         |  USB  |
                         | key |char |  key  | char |  loc  | Usage |
   keydown      Shift       16    0    Shift   null   LEFT   0x700E1
   * keydown    Shift       16    0    Shift   null   LEFT   0x700E1
   keydown        2         50    0      @       @      0    0x7001F
   keypress                 64   64      @       @      0
   * keydown      2         50    0      @       @      0    0x7001F
   * keypress               64   64      @       @      0
   keyup        Shift       16    0    Shift   null   LEFT   0x700E1
   * keydown      2         50    0      2       2      0    0x7001F
   * keypress               50   50      2       2      0
   keyup          2         50    0      2       2      0    0x7001F


Appendix B - USB Usage Examples
===============================

   USB    USB
  Usage  Usage   Windows
   Page    ID   Scancode  Name
   0007   0004     1e     Keyboard a and A (qQ on French keyboard)
   0007   0005     30     Keyboard b and B
   0007   0006     2e     Keyboard c and C
   ...    ...     ...     ...
   0007   001c     15     Keyboard y and Y (zZ on German keyboard)
   ...    ...     ...     ...
   0007   00e0     1d     Keyboard LeftControl
   0007   00e1     2a     Keyboard LeftShift
   0007   00e2     38     Keyboard LeftAlt
   0007   00e3     5b     Keyboard Left GUI (Win or Cmd key)
   ...    ...     ...     ...
   000c   00b5    e019    ScanNextTrack
   000c   00b6    e010    ScanPreviousTrack
   000c   00b7    e024    Stop
   000c   00b8    e02c    Eject
   000c   00cd    e022    Play/Pause

A more complete mapping table between USB usage codes and
platform-specific scancodes can be found in the Chromium source code
at:
http://code.google.com/searchframe#OAMlx_jo-ck/src/ui/base/keycodes/usb_keycode_map.h


References
==========

Document Object Model (DOM) Level 3 Events Specification
W3C Working Draft 14 June 2012
http://www.w3.org/TR/DOM-Level-3-Events/

Universal Serial Bus HID Usage Tables
Version 1.11 6/27/2001
http://www.usb.org/developers/devclass_docs/Hut1_11.pdf

Received on Friday, 7 September 2012 18:38:11 UTC