Re: Way forward and IME behavior speccing

On Tue, Oct 13, 2015 at 9:31 AM, Piotr Koszuliński <
p.koszulinski@cksource.com> wrote:

>
> On Mon, Oct 12, 2015 at 5:42 PM, Johannes Wilm <johannes@fiduswriter.org>
> wrote:
>
>>
>>
>> Well, as mentioned "Consider re-conversion, that turns existing text into
>> composition" [1], on Android, one can click on a word and then that word is
>> moved back into the IME composition and can be edited there. So the outcome
>> of that operation when the composition is finished is that word A has been
>> turned into word B, ie "edited".
>>
>> This seems to also work if word A is a more complex HTML structure, such
>> as "<b>A<b/>pple". But trying this out on TinyMCE and CKeditor, it simply
>> seems to replace the entire thing with a string "Apple", even if the user
>> just clicks on the word once without editing any of it. Not sure how much
>> this is cE itself and how much is CKeditor/TinyMCE.
>>
>>
>
> That's done solely by Safari (I checked Safari@iOS). CKEditor isn't
> interfering here (see http://jsfiddle.net/r6y18wLa/).
>
> BTW. If I tap "<p>A<b>p</b>le</p>" and choose the autocorrect option I end
> up with "<p>Apple</p>". If I had "<p><b>A</b>ple</p>" I end up with
> "<p><b>Apple</b></p>".
>

The same happens on Android, except that I don't even have to choose
autocorrect. I just click on the word, it is being underlined and the
formatting has been changed. Then I click somewhere else (not having edited
the word) and the changed formatting is not being reverted.

So based on that it seems both the IMEs on Android and iOS are "messing
with the DOM".

What about something more complex where I add certain extra information? As
to be expected, "<p><b class="first-letter">A</b>ple</p>" turns into "<p><b
class="first-letter">Aple</b></p>" just by clicking on it, without editing
anything. [1].

This is all very problematic.

I imagine that the IME itself is not aware of HTML, and that it's the
"glue" between IME and browser that does the actual change in the DOM, but
it is an uncontrolled change nevertheless. Just as with spell checking, the
browser makes some assumptions about what is a word, across element
boundaries. and it thinks it knows that it is OK to mess with the DOM,
making elements entirely disappear or enclosing all letters the browser
thinks belong to one word within one element even without the user having
done any editing operation (on Android).

So I see two choices here:

A) In this new mode we continue to entirely forbid IMEs to deal with more
than just the contents of a text node. If the IMEs are unaware of DOM
structures, then surely this must be in the browser code that glues it to
IMEs. The user experience will be slightly worse for a few cases (auto
correct will not work on partially styled words), but it will be a major
improvement for all other cases and it will do that JS editors will not
just break randomly when dealing with IMEs.

B) We make all of these IME-introduced changes preventDefault'able. IMEs
will just have to be able to handle this if they want to be standards
compliant. Also, may not a lot of this be doable creatively in the
glue-code? If the IME doesn't allow preventDefault, but the default action
is canceled in JS, couldn't the glue code simply abort the current IME
operation (tell the IME that composition has been finished) and then start
it again immediately thereafter and starting out with the text as it is
after the JS defined operation. To the JS code that would make it look as
if IME input would be preventDefaultable.

Additionally, I think we need for IME input to be sandboxable - either by
the browser handling the input in a Shadow DOM itself, or by letting the
JavaScript move the caret to a ShadowDOM. Or by means of some of other
mechanism that the browser makers may want to suggest.


[1] http://jsfiddle.net/r6y18wLa/1/



-- 
Johannes Wilm
Fidus Writer
http://www.fiduswriter.org