Re: Way forward and IME behavior speccing

> On Oct 11, 2015, at 11:29 AM, Johannes Wilm <johannes@fiduswriter.org> wrote:
> 
> On Sun, Oct 11, 2015 at 4:48 PM, Koji Ishii <kojiishi@gmail.com> wrote:
>> I'm not sure we're moving towards to the right direction.
>> 
>> I wasn't in the discussion, but what I remember was that the TF moved from ce=events to ce=typing because with events, JS has to handle all IMEs and composited characters, and Ben said that JS developers wanted to avoid that.
> 
> Ok, so for those new to the discussion, it went something like this:
> 
> Koji and Ben Petersen (at that time at Microsoft) were discussing ways of making IME input transparent to the browser.
> 
> Ben left, and I joined this part of the conversation. The browser people insisted that IMEs cannot be controlled in any way and that one cannot cancel their insertion of a letter or otherwise. So I asked if at least we could sandbox it. Koji mentioned we could extend text nodes to hold a shadow root and do the editing process transparently inside of that. Ryosuke (Apple) then said that was impossible. I then proposed we could just use the lowest level element node, copy a text node and all its siblings into it, then do the editing in that shadow root, all transparent to the user. Ryosuke and Koji seemed to agree.
> 
> Then we met in Paris. Ojan (Google) said that doing this kind of shadow root type handling was non-performant, but instead he offered that we could write the spec so that moving the care to a newly created shadowDOM and do the composition there could be handled in JavaScript. The editor people and everyone present seemed happy with that. I wrote a small piece of Javascript to try out whether this is actually working. It turns out it was working on Chrome, But it was somewhat inconsistent, so I went and talked to the Ui events group in the hope of getting standardization of it. Koji and some others were there and once they heard what I was doing (moving the caret during compositionstart), they wondered whether the spec should be changed to forbid me from doing this. Among other things, a new and shiny IME in Windows 10 is very fragile and apparently breaking this, so the spec would possibly be needed to readjusted.
> 
> While it seems somewhat frustrating that there seems to be so little understanding for why we need to spec anything at all, and why we need JS editors, and why can't we just leave it all up for the browser, when browsers have failed to advance in this area for 15 years and have no comprehensive plan of their own to make it work, I guess these are the types of conflicts one will run into any time one starts touching an area that involves many people.
>>  
>> 
>> If ce=typing needs to define and handle all IME related complications, it looks to me that we lost something. Not sure what we missed, but my preference is to discuss to find a way without defining everything related with IME.
>> 
>> Do you have a list of issues why you need to handle IME, so that we could try to find a way without defining it?
> 
> 
> Among the issues are:
> 
> 1. Keep all changes atomic. This makes it much easier to handle in terms of history and collaboration handling. One way of doing this is to sandbox the IME, then figure out what the changes are when it is done (my approach). Another approach is to just let the composition happen in the DOM and onc eit is done, figure out what it has done and potentially role that change back and then create an atomic change.
> 
> 2. Just like editors don't want contentEditable to change the DOM, they don't want IME to do it either. The editors want and need to be in charge of what actually happens in the editor in the end. The "damage" that browser in editable mode can do to the editing process can in most cases either be directly canceled or otherwise immediately be "rolled back" by diffing and patching the DOM. This is not the case for IME input, as the composition process only can be interrupted on a limited and somewhat unpredictable way. So at the very least one needs to let the IME finish the entire composition, before one can roll the changes it has made to the DOM back and instead change the DOM the way we really need to.
> 
> While IMEs mostly just manipulate the contents of text nodes, the future of IMEs is apparently a lot more complex. As I believe you mentioned yourself, Koji, IMEs on Android edit entire words, even if the word has more complex DOM structures inside of it.
>  
> 
> For example, say I have a language in which "2nd" is written as "2<sup>a</sup>". So the user types "2" and "a", and the IME then automatically inserts:
> 
>  2<sup>a</sup> 
> 
> or because it is another browser it may decide to add this type of HTML within the IME:
> 
> 2<span style="font-size: 11px;font-face: Arial;vertical-align:sup;" class="--Webkit-custom-property">a</span>
> 
> or some such thing. Because it's IME, one won't be able to cancel it or change it, at least not before the entire composition is done.
> 
> So, to stop all of this, we need a clear specification of IMEs. And it needs to be restrictive. There are a million things IMEs might do, but if every IME does it's own thing, it's bound to just break most of the internet and not to be of benefit to anyone.

The problem is that we can't. IME is a system component. Browsers can't modify how IME works on each system.

> For example, I notice that with the only multi language keyboard on Android 5.1 (Multiling O Keyboard) with Chrome on standard sites such as Facebook, correcting previous comments is just fundamentally broken. In essence, the argument for why IME behavior needs to be standardized is the same argument for why anything needs to be specced and standardized and why we cannot just have 4-5 different browsers, each doing it's own thing in a different way.

What we need is an abstraction which hides all these platform-dependent IME behaviors.  This is precisely why we've advocated for shadow-DOM like solution.  So that IME can do whatever it wants to without modifying the real DOM.

On the contrary to what Ojan said during the meeting, I don't think we can make this new API completely backwards compatible with contenteditable in regards to IME if we wanted to accomplish our goal of browser not messing with DOM at all.

- R. Niwa

Received on Monday, 12 October 2015 00:53:28 UTC