Android IME Testing

Hey everyone,
I hope to see at least some of you tomorrow. The first debate will be on
whether we should continue building the Input Events spec as planned, or
whether we should try to work on alternative approaches, as Alexandre
proposed.

In connection with that, I repeated the test I did half a year ago of
Android keyboards [2]. Half a year ago I tested different Android
keyboards. Today I tested them again. The idea was to see if it would be
possible now to build an editor based on the various events coming out of
the IME process. I have only tested a limited number of very basic editing
operations. The findings are in short:

* None of the more popular keyboards work according to spec. Hacker's
Keyboard seems to be an exception.
* Each keyboard gives different events for the same action - most of them
are broken.

This leads me to the following conclusions:

* Given the current state of IME on Android, it's unrealistic for JS
developers to rely on these events. The only reasonable choice is therefor
to monitor DOM changes.

* Even if keyboards were fixed to work, the events that are triggered
differ from keyboard to keyboard. If JS devs choose to work with this, they
would need to target not only the browser but each individual keyboard.

* By not implementing the Input Events level 2, Chrome saved 2 DOM updates.
Yet instead it is sending 16 events rather faulty events when the user hits
the Enter-button. Any gain in speed obtained by saving those two
DOM-updates should be long gone by the JavaScript code that needs to be run
and will update the DOM to handle with these 16 events.

I realize that some will say that IME is part of the user agent and
therefore out of our scope. However, given that JS devs need to target
individual keyboards, it ends up being our concern anyway. Maybe an
advantage by having keyboards work so differently is also that apps should
not care much if you were to standardize keyboard behavior and the events
that are triggered?


For this test, I used the a slightly modified version of uievents key event
viewer in that I replaced the input field with a contenteditable element
[2].


Gboard:

*If I type a single letter in a pre-existing word, I get this series of
events: *

*compositionend - keydown - beforeinput (insertText) - input (insertText) -
keyup. *

Notice: lack of compositionstart

*If I type at the end of the existing text, I get:*

*keydown - compositionstart - beforeinput (insertCompositionText) -
compositionupdate - input (insertCompositionText) - keyup*

Correct

*If I try to select one of the available choices after typing one letter:*

*compositionend - keydown - beforeinput (insertText) - input (insertText) -
keyup*

Notice: lack of compositionstart

*If I try to hit enter in the middle of an existing work, I get 16 (!)
events:*

*compositionend - keydown - input - input - keyup - keydown - beforeinput -
input - input - keyup - keydown - compositionstart - beforeinput -
compositionupdate - input - keyup *

Notice how the sequence starts with a compositionend, how there are input
events without corresponding beforeinput events and how there are three
keydown-keyup combinations where really only one key was pressed.

*If I try to hit backspace in an existing word, I get:*

*compositionend - keydown - input - keyup*

Notice the compositionend without a compositionstart and the input without
a beforeinput



Fleksy:

*If I type a single letter in a pre-existing word, I get this series of
events: *

*keydown - beforeinput (insertText) - input (insertText) - keyup. *

Correct

*If I type at the end of the existing text, I get:*

*keydown - beforeinput (insertText) - input (insertText) - keyup. *

Correct

*If I try to select one of the available choices after typing one letter:*

*keydown - input (deleteContentBackward) - keyup - keydown - beforeinput
(insertText) - input (insertText) - keyup*

Notice how this is deleting the letter and then reinserting the entire word
wheras the gboard just added the missing part of the word. Notice also the
missing beforeinput event.

*If I try to hit enter in the middle of an existing work, I get:*

*keydown - keypress - beforeinput (insertParagraph) - input
(insertParagraph) - keyup *

Correct (yet interesting that there is a keypress event here)

*If I try to hit backspace in an existing word, I get:*

*keydown - beforeinput - input - keyup*

Correct


Multiling O:

*If I type a single letter in a pre-existing word, I get this series of
events: *

*keydown - compositionstart - beforeinput (insertCompositionText) -
compositionupdate - input (insertCompositionText) - keyup. *

Correct

*If I type at the end of the existing text, I get:*

*keydown - compositionstart - beforeinput (insertCompositionText) -
compositionupdate - input (insertCompositionText) - keyup. *

Correct

*If I try to select one of the available choices after typing one letter:*

*keydown - beforeinput (insertCompositionText) - compositionupdate - input
(insertCompositionText) - compositionend - keyup - keydown - beforeinput
(insertText - '' ') - input (insertText - ' ') - keyup *

Notice how a composition is ended and then a space-string is inserted

*If I try to hit enter in the middle of an existing work, I get:*

*keydown - keypress - beforeinput (insertParagraph) - input
(insertParagraph) - keyup *

Correct (yet interesting that there is a keypress event here)

*If I try to hit backspace in an existing word, I get:*

*keydown - input (deleteContentBackward) - keyup - keydown -
compositionstart - beforeinput (insertCompositionText) - compositionupdate
- input (insertCompositionText) - keyup*

Notice the missing beforeinput and the incorrect insertCompositionText
events


Hacker's Keyboard:

*If I type a single letter in a pre-existing word, I get this series of
events: *

*keydown - beforeinput (insertText) - input (insertText) - keyup. *

Correct

*If I type at the end of the existing text, I get:*

*keydown - compositionstart - beforeinput (insertCompositionText) -
compositionupdate - input (insertCompositionText) - keyup. *

Correct

*If I try to select one of the available choices after typing two letters:*

*keydown - beforeinput (insertCompositionText) - compositionupdate - input
(insertCompositionText) - compositionend - keyup - keydown - beforeinput
(insertText - '' ') - input (insertText - ' ') - keyup *

Notice how a composition is ended and then a space-string is inserted

*If I try to hit enter in the middle of an existing work, I get:*

*keydown - keypress - beforeinput (insertParagraph) - input
(insertParagraph) - keyup *

Correct (yet interesting that there is a keypress event here)

*If I try to hit backspace in an existing word, I get:*

*keydown - ** beforeinput (deleteContentBackward) -** input
(deleteContentBackward) - keyup*

Correct


[1] http://lists.w3.org/Archives/Public/public-editing-tf/2017Jun/0004.html
[2] https://johanneswilm.github.io/uievents/tools/key-event-viewer.html

-- 
Johannes Wilm
Fidus Writer
http://www.fiduswriter.org