Re: 2 Proposals for Minimum Viable InputEvent

On Mon, Feb 6, 2017 at 6:18 PM, Johannes Wilm <johannes@fiduswriter.org>
wrote:

>
>
> On Tue, Feb 7, 2017 at 12:23 AM, Alexandre Elias <aelias@google.com>
> wrote:
>
>> Hi, I'm the maintainer of IME on Android.  I'll be happy to attend the
>> conference call next week.  As you've surmised, Android does need to
>> perform a DOM change to merge two nodes "later".  However, I disagree that
>> those 5 steps are somehow needed.  Our plan is to send a single
>> insertCompositionText beforeinput event instead of 5 events.
>>
>
> Nice to meet you Alexandre!
>
> It's a bit unfrotunate that you haven't been able to participate in the
> last 3 years of discussions about this subject (I think some people have
> even been discussing this for more than 8 years), and it seems that the
> mailing list isn't as alive any more as it once used to be, but I'll do my
> best to give the reasoning why we came to the conclusion that these events
> are needed. I would also recommend consulting the various open and closed
> issues on github and to consult the archive of the mailing list. Also,
> various Google people have been behind a lot if not most of the various
> various concepts, so they can probably give you more background about this
> as well.
>
>
>>
>> Arguments why each is not needed:
>>
>>  > 1. deleteByComposition --This step is only needed when recomposing. A
>> composed word can go across element boundaries, so this may delete more
>> than just contents of a text node. (fx " He<b>ll</b><i>o fish</i>!"). This
>> step needs to be executed at the moment your Android IME changes the DOM,
>> whether this is at the moment the user taps the word and the underline
>> appears or at a later stage.
>>
>> There's no need for a separate "delete" event, because
>> insertion-to-replace-range is general enough to represent a deletion range
>> as well.  I prefer to reserve "delete" events exclusively for intended IME
>> deletions (e.g. select word and press backspace).
>>
>
> But I am a little confused: From the proposal Chong sent in, it sounded
> like you just wouldn't have any beforeinput event for this initial
> deletion. That would create a range of different problems, such as:
>

I agree we should not hide any editing information from JS.  I am saying
that event 2/3 can encompass the deletion.  An implied, wrong assumption
underlying your argument seems to be that an insertion operation can only
ever add new text but not remove existing text.  In reality, insertions are
always *replacements* of the content of the caret/selection/composition
with text (for example, if you select text with the mouse and then type "a"
on a physical keyboard, no deletion event is sent).


> * The more advanced editors today are moving toward a model where they
> hold a json version of the document which is separate from the DOM. They
> only make adjustments to the DOM when needed, and if the user changes
> something about the DOM, they make adjustments to the  json document. The
> json version is the authoritative copy of the document. For this to work,
> they need to know about any change to the DOM. If you make user-initiated
> DOM changes that are not causing any beforeinput event at all, the
> beforeinput event is somewhat useless, aand they will have to continue to
> monitor user initiated DOM-changes in other ways.
>
> * In a collaborative editor, if you make a change to the DOM of one user,
> and the JS is not being made aware of this change, then it won't notify the
> collaborators of this change, and so the collaborator will ahve different
> versions of the document. Again, the point of beforeinput events will be
> gone. JS develoeprs will use mutation observers or some other technology
> instead.
>
>
>
>
>>
>> > 2. insertCompositionText -- Also this step only needs to happen at the
>> moment only for recompositions. It reinserts the word from step one, but
>> this time as plaintext in one element (fx "Hello" => "<i>Hello fish</i>")
>>
>> On Android because of the "late" timing, this merge operation would
>> happen in the same message loop task as step 3, so there is no need for
>> this event.  We prefer to go directly from "He<b>ll</b><i>o fish</i>" ->
>> "<i>Help<i>".
>>
>
>
> Yes, in a perfect world, where all IME-related events are cancelable, this
> could be done and would be preferable, as for example Yosin has pointed out
> ( https://github.com/w3c/input-events/issues/45#issuecomment-261427648 ).
> We had a long discussion during the fall of 2015 about this, and among
> other ones several developers from Chromium eventually convinced us here
> that IME-events cannot be guaranteed to be cancelable. Even moving the
> caret to somewhere else during the composition may somehow crash the IME
> (hwo this is possible is a bit of a mystery to me as a JS developer).
>
> One could then say ok, combine 1 and 2 and make them both non-cancelable.
> The problem with that is that the browser doesn't always know the seamntic
> meaning of the various DOM elements used within a document and it may end
> up making decisions of what to delete that are problematic.
>

I've been trying to figure out where these claims about IME from some
Chromium team members came from exactly and haven't figured it out, nobody
has a precise memory.  They seem to be based on a vague fears rather than
concrete knowledge of how IMEs behave.  I agree with your instinct that the
assumptions behind it don't make too much sense, that's why I think we
should hold off before we commit to a complicated model based on those
assumptions.


>
>
>
>> However, rniwa@apple.com has told us that on OS X, the standard platform
>> behavior is to merge the nodes at time of composition start.  Therefore, on
>> OS X step 2 is still needed and there would be 2 beforeinput events in
>> total, but on Android there would be only 1.  This represents a substantive
>> platform difference that I think it makes sense to allow to differ in the
>> spec.  I don't think we should try to artificially synchronize the
>> platforms by always sending 2 events -- on Android this would simply result
>> in unnecessary work and side effects that would never be seen by the user.
>>
>
> I think you should instead consider making this two events on Android.
> Android is really great, but text editing in Chrome on Android is really
> problematic especially when it comes to IME recomposition, as I am sure you
> are aware. I have tried various keyboard, and but even on mainstream sites
> such as Facebook, using a Google-provided keyboard, recomposition tends to
> just mess up entire sentences, throw random words around, deleting or
> merging words that are close to what one is trying to recompose, etc.
>

Right, we are in process of fixing all of these bugs and expect to get the
remaining ones fixed in M58/M59 timeframe.  Anyway, this doesn't address my
point.  Personally, I believe that firing additional unnecessary events
will prove harmful to both UAs and JS editors -- in reality an atomic IME
change that just happened, but you are proposing to drip-feed it into many
separate events.  Your proposal doesn't provide JS any additional
information (it just splits information into pieces), so therefore I don't
think it can possibly be needed for any use case either.



> Speed is not as much a concern anyway because it all depends on the human
> who is interacting with it, but in addition these really major issues mean
> that recomposition as it exists on Android as of today is not very useful..
> In addition there are the various issues that JS developers have complained
> about for a while: no compositionstart event, etc. . I am sure you know the
> list of grievances with it all too well.
>

We should always be firing compositionStart today.  Feel free to file a bug
against me if you know of a case where we don't.


>
> Not knowing the details, but I could imagine you can check for event
> listeners, and only if those are present you make it two events.
>
> But as an alternative, and to try to get a bit closer to both your and
> Yosin's position, how about this:  We make two IME models, one for IMEs
> where the beforeinput events can be cancelled in which step 1 and 2 are
> merged and one where they cannot be cancelled where they are separate.
>

Disagree, we should keep things as simple as possible.  I want us to fire a
minimum of events and avoid complicated logic forking.  I don't want us to
introduce permanent complexity into the platform for the sake of a
short-term compromise.



>
>>
> > 4. deleteCompositionText -- removed (fx "help"): It removed the text
>> from the underline and it may not be the entire composition.
>>
>> > 5. insertFromComposition -- adds "help" permanently, and gives the JS
>> the opposrtunity to rebuild the complex HTML structure lost in step 1.
>>
>> Step 3 is already permanent.  The fact that there is still an active
>> composition underline at the time of step 3 does not make it somehow
>> temporary.  It has already been injected into the DOM and caused all side
>> effects.  So the JS should rebuild its complex structure at time of event
>> 3, and events 4/5 are redundant.
>>
>
> When a composition starts, they temporarily pause most activities because
> they are not really allowed to intervene. Then, once the composition is
> finished, they deal with the finished text.  input. For example, in step 5
> they may reapply some partial styling that was lost in step 1.
>

"compositionend" event is fired at the same timing that steps 4/5 would be,
isn't it sufficient for this use case?


>
>
>
>>
>>>
>>> The reason you state for the proposed change is that "On Android it’s
>>> very common for IME to start composition whenever a cursor is moved onto a
>>> word.  This is primarily intended to passively show an underline to the
>>> user (to indicate what would be replaced *if* user selects an alternate
>>> spelling)."
>>>
>>> If such tapping leads to actual DOM changes, then  the JS needs to know
>>> about those(step 1/2). But if you don't change the DOM immediately and only
>>> do so at a later stage when you "really" start the composition, then you
>>> can also wait until that stage.
>>>
>>>
>>> So these steps are needed on allplattforms, no matter whether or not you
>>> change how IMEs work on Android.
>>>
>>>
>>>>
>>>> Adding aelias@ to make sure he is able to make the call on the 14th.
>>>>
>>>> dave.
>>>>
>>>> On Fri, Feb 3, 2017 at 1:33 AM, Johannes Wilm <johannes@fiduswriter.org
>>>> > wrote:
>>>>
>>>>> Could we get the person or people who came up with these changes due
>>>>> to Android to participate in the call on the 14th? I think it's very
>>>>> important to have them invovled in this discussion.
>>>>>
>>>>> On Fri, Feb 3, 2017 at 10:21 AM, Johannes Wilm <
>>>>> johannes@fiduswriter.org> wrote:
>>>>>
>>>>>>
>>>>>> On Fri, Feb 3, 2017 at 10:18 AM, Johannes Wilm <
>>>>>> johannes@fiduswriter.org> wrote:
>>>>>> ...
>>>>>>
>>>>>>> Right now I cannot see any purpose of the beforeinput event with
>>>>>>> these changes applied, but maybe I am missing something?
>>>>>>>
>>>>>>>
>>>>>> I need to restarct that. I guess it could still be useful to stop
>>>>>> native bold/italic buttons from making their own, non-controlled DOM
>>>>>> changes. It's just for text input that it is irrelevant. And IME will
>>>>>> continue to be a mess.
>>>>>>
>>>>>> --
>>>>>> Johannes Wilm
>>>>>> Fidus Writer
>>>>>> http://www.fiduswriter.org
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Johannes Wilm
>>>>> Fidus Writer
>>>>> http://www.fiduswriter.org
>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Johannes Wilm
>>> Fidus Writer
>>> http://www.fiduswriter.org
>>>
>>
>>
>
>
> --
> Johannes Wilm
> Fidus Writer
> http://www.fiduswriter.org
>