W3C home > Mailing lists > Public > public-canvas-api@w3.org > July to September 2010

Re: html5 editor responds to Canvas accessibility related bugs

From: Oliver Hunt <oliver@apple.com>
Date: Wed, 29 Sep 2010 15:53:28 -0700
Cc: Ian Hickson <ian@hixie.ch>, public-canvas-api@w3.org, public-canvas-api-request@w3.org
Message-Id: <E8B5C9D6-ACDD-448C-8E5C-1F970686B551@apple.com>
To: Charles Pritchard <chuck@jumis.com>
Charles sent me a email directly, and i'm unsure if he was okay with me forwarding it, so I am not, but as background for

This isn't simply a question of layout, it's a question of text _entry_, specifically the mechanisms of how this works, the most simple are dead characters but the range to complex selective and context aware input methods such as those used for japanese, traditional and simplified chinese, etc.  Text entry for even the most simple of scripts does not necessarily correlate the keyCode or charCode to the actual character that is expected to be shown.

For example the default input managers for many of the latin scripts (english, french) on Mac OS makes use of dead characters for accents and alternate character selection, and windows lets you install different input managers (or installs by default if you're in the appropriate locale) for similar features, although the input mechanism and ui is substantially different.

A very simple example of how much people overlook is the character 'ß' which on mac, with the default input method, is achieved through option-s, and yet it doesn't work in any hand rolled editor that i have seen.  You could argue this is the specs fault, but the underlying problem is that text entry is not trivial, there is not necessarily a one-to-one correspondence between keys pressed and characters displayed, it is an M-to-N relationship and simply dealing with what the final displayed text should be is difficult.  As a further example, while ß is a "single" keystroke, accented letters may use dead characters, eg. é is achieved through option-e, e where option-e produces a "dead character" for the incomplete accented character.  Once again this behaviour is only with the default input manager, for people in certain locales, on Mac OS.  The windows and linux equivalents are similar in behaviour for some of these characters, but the actual key sequences differ.

This also ignores the user visible UI, which agains differs from OS to OS, language to language, input method to input method.  I'll now try to illustrate the range and complexity of text entry using a pile of screenshots.

Default Mac IME, typing option-e.  The yellow highlight and the text shown are generated by the IME, the US doesn't have specific knowledge of what is going on:


Using the Mac Kotoeri IME in hiragana mode, typing the keys <t> <o> <down arrow> <enter><enter> produces the following UI
(type t)(type o)(press down arrow)  (press enter) (press enter) 

Typing those same characters when in katakana mode of the produces the character トalthough these input methods need to have access to the content being edited as they adjust there behaviour according to context as well.  It gets even better, as we can get multiple composition regions and you can move through and change each composition region, eg.

toukyouhaame becomes とうきょうはあめ but you can modify the composition as you type producing UI that allows you to move through different sections of the IME (bold underline is the active portion of the composition, in _this_ IME, this is not necessarily true of any other)


Your final text may be とうきょうはあめ or it might be 等kyouhaアメ even though the key events you receive may be exactly the same.

Typing those same characters again, using the simplified chinese ime included in macos (in pinyin mode) produces the following UI:
(type t)

(type o)

(press down arrow)

(press enter)

(press enter)

There are 3 other modes (with different UI) in the simplified IME, then there's an IME for traditional chinese with 4 modes, and another different UI.

This a small sample of just a few of the different IMEs that come by default with MacOS, there are multiple 3rd party IMEs for many languages, some of which are very popular, many with completely different behaviours, some with their own distinct UIs, etc.  Again this is only MacOS, Windows has its own IMEs, and has many third parties IMEs there as well, and they have different UIs and behaviour as well, and Linux has its own as well.

Because of these huge differences between the IME models from one OS to another it is not possible to produce a complete and sane API that would allow JS to interact with these correctly, and even if it were, a huge amount of work is needed to get correct behaviour as you are required to communicate a large amount of information with the IME, and the IMEs require a very rich interface in order to correctly control what the user can see.  After all of that there's also the potential for information leakage and other security issues (phishing, etc) that would make me very reticent about allowing JS to control data being fed to the IMEs.

My hope is that by making the complexity of text input visible to the people reading this thread you can understand why I am opposed to APIs that attempt to make it possible to create editors using canvas -- it is simply not possible to create an API that allows someone to create a fully fledged editor, and any attempt is simply going to result in a half-assed API that only works "mostly" for people of the english (or similar scripted) speaking/writing world.

Rather than saying there are legal obligations that require you to create an accessible interface for a editor written in canvas, you should be asking what it is that has resulted in the need to roll your own editor in the first place.  Then request appropriate features in the browsers directly, as the browsers already correctly manage user input, are able to interact with them safely, and ensure that any IME UI are correct.

--Oliver
Received on Wednesday, 29 September 2010 22:57:48 GMT

This archive was generated by hypermail 2.2.0+W3C-0.50 : Wednesday, 29 September 2010 22:57:49 GMT