Re: 2 Proposals for Minimum Viable InputEvent

Hey,

as this was getting quite complex, I have tried to list all the proposals I
could find in one document and I have given editing access to those who
came up with proposals so far. Let me know if you need write access.
Otherwise, ehre is comment access for eveyrone else:
https://docs.google.com/document/d/1uE3Whzp_zlGBh3giBtyJYcHm3hdTC-PVeyDMFsBerF8/edit?usp=sharing

In terms of beforeinput events during IME, I am currently in favor of
Alexandre's proposal A5 (next alternatives: A4 or no change)

In terms of cancellable beforeinput event, I am in favor of no change. I
don't think I have seen any reasonable argument for why we should make
anything except IME non-cancelable.

On Fri, Feb 10, 2017 at 11:56 PM, Johannes Wilm <johannes@fiduswriter.org>
wrote:

>
>
> On Wed, Feb 8, 2017 at 3:59 PM, Piotr Koszuliński <
> p.koszulinski@cksource.com> wrote:
>
>> Hey,
>>
>> I won't try to go too deep into how IME related events should work
>> because I don't know IME mechanisms and all its variations well enough,
>> despite working with cE for more than 5 years. Perhaps, this answers some
>> questions like how hard IME is actually and what's the average knowledge
>> about it, even amongst quite experienced RTE developers ;).
>>
>> Anyway, I'd like to rather focus on how non-preventable the
>> beforeinput event can be to make any value for CKEditor 5. I hope it will
>> provide some context for the discussion.
>>
>>
>> ### Text insertion ############
>>
>> A couple of months ago I created a PoC of CKEditor 5's typing feature
>> working based on the beforeinput event available at that time in Blink [1].
>> Unfortunately, I can't quickly check how the editor would behave if the
>> event wasn't cancelled, but that would be very similar to how the editor
>> needs to work without the beforeinput event.
>>
>> So how it works now?
>>
>> 1. We use a mutation observer to catch the input (typed characters).
>> There's a special case when selection isn't empty when the user starts to
>> type. We needed to hardcode the unsafe keys and delete the selected content
>> on keydown. Otherwise, typing a single letter might cause hundreds of
>> mutations if selection crossed some complicated content. We must avoid that.
>> 2. We diff the DOM with our virtual DOM implementation to get a
>> simplified form of the mutations. Browser may fire dozens of them, while
>> for us it can be just a few changed text nodes or  removed elements.
>> 3. Then we analyse the mutations. We try to find a fragment of removed
>> and inserted text, plus, the position of the selection after the change.
>> 4. Then, if a feature (in this case, the typing feature) can handle
>> certain mutations (i.e. they were not too complex) it applies the necessary
>> changes to the model.
>> 5. The changes in the model are converted to the virtual DOM.
>> 6. If virtual DOM differs from the real DOM, minimal changes are applied.
>>
>> This architecture allows us to:
>>
>> * Apply understandable changes to the model, without re-rendering the
>> DOM. This is crucial to support IME because some changes in the DOM break
>> the composition (see the "IME" section in [1]).
>> * Re-render the DOM before paint happens if changes were rejected by the
>> editor. That, fortunately, needs to happen very rarely, but it's nice that
>> there's no flickering anyway.
>>
>> How would beforeinput simplify our implementation?
>>
>> * We wouldn't need to listen to mutations and normalize them (1. & 2.).
>> * We wouldn't need that ugly unsafe keys handler meant to empty the
>> selection before a character is inserted (1.).
>> * We would avoid the diffing mechanism, which is quite complex already
>> despite missing few edge cases with spell checking or auto correction (3.).
>> * We would have a perfect clarity whether the inserted space was meant to
>> be a space or non-breaking space (it's problem to learn that when it's at
>> the end of a block).
>> * Etc, etc.
>>
>
>
> Shouldn't you have the same information iof you listened to all the
> compositionupdate events (and these events actually delivered accurate
> data)?
>
>
>>
>> So, that's the profit even if we can't cancel the event. It's still
>> a pretty significant simplification, comparing to what we have to do now.
>>
>> However, there's one nasty case which may still require hacking if the
>> event isn't cancelled. It's how the editor can delete the content itself
>> when the user is typing over a non-empty selection. We can't let the
>> browser do that because we wouldn't know how to convert those changes to
>> the model (and sometimes it might simply be impossible). In general, there
>> are a couple of options here.
>>
>> * If we'd be able to cancel all beforeinput events, that would be
>> perfect, because we'd be able to handle the deletion of the content by
>> ourselves first, and then we'd insert the typed characters.
>> * If the beforeinput event is not cancellable, then:
>>    * it'd be nice if for the non-empty selection there would be
>> a beforeinput with type=deleteContent fired before beforeinput with
>> type=insert* (that would natively split the process into two steps –
>> deletion and insertion),
>>    * alternatively, we should be able to change the DOM
>> on beforeinput event without breaking the native insertion itself (then we
>> can simply delete the selected content and set selection in a correct
>> place, where the new character should be inserted).
>>
>> One of these options must be true because, otherwise, we won't be able to
>> use the event. We'd still need to base on some ugly heuristic for
>> discovering keys which are about to delete the content of the selection.
>>
>> What I don't know is how these options apply to IME. Let's assume that
>> its events are non-cancellable, so we get some beforeinput event of delete*
>> type followed by insertCompositionText. The deletion part is unclear for me
>> because the insertion might've been in a middle of composition (so you get
>> deleteCompositionText + insertCompositionText) or it might be the first
>> composed character. What do we get in this case?
>>
>> I'm asking because I assume that when the user is composing already, the
>> browser will never do some significant DOM changes. Composition would
>> normally only touch the text nodes in a certain block, but not the block
>> itself. Besides, while composition takes place, we're not able to re-render
>> the DOM anyway, because that breaks the composition (see the "IME section
>> in [1]). So, if something is wrong, we need to wait for the composition to
>> finish anyway. But this is acceptable because I don't expect significant
>> changes to be required. Maybe just some inline styling.
>>
>> However, if the insertion starts the composition, then the content which
>> was previously selected might contain multiple blocks, tables, etc. and we
>> must handle deletion of that piece ourselves. So, at least in this case, we
>> must be able to either act upon inserComopositionText or there must be a
>> separate deleteContent event.
>>
>> Uhh... that was about text insertion.
>>
>>
>> ### Other beforeinput types ##################
>>
>> A non-cancellable beforeinput for the text insertion (and text deletion)
>> is acceptable (with couple limitations, as I tried to explain above).
>> However, other before input events must be cancellable. Otherwise, there
>> will be quite pointless.
>>
>> For example, how do we handle the backspace and delete keys now? We
>> listen to keydown, discover which key was pressed and with what modifiers
>> and delete the content. We might not be able to implement "delete word" or
>> "delete soft line" very precisely, but these are really minor issues. So,
>> in this case the before input wouldn't give us anything. Actually, it'd be
>> even worse than listening to the keydown event because we can cancel the
>> keydown event.
>>
>> The same with text formatting – we can easily listen to CTRL+B to know
>> whether someone wants to bold a text. We can cancel the keydown event. It'd
>> be much worse if we had to listen to the beforeinput event and fix the DOM
>> broken by the browser which incorrectly applied some <b> tag. The
>> beforeinput event would only help with localizing the keystroke (in some
>> languages it won't be CTRL+B).
>>
>> And so on, and so on.
>>
>> I can understand that IME makes some beforeinput events non-cancellable.
>> But I totally don't understand why would deleteContentForward or
>> insertUndeorderList be non-cancellable too (as was proposed in the document
>> [2]). The set of non-cancellable events should be minimal. Otherwise, the
>> situation is not going to improve.
>>
>>
>>
>> [1] https://github.com/w3c/editing/issues/149
>> [2] https://docs.google.com/document/d/1yPZEkHl_WOPjVeilZjE1
>> XmlyjLCe-uS7THI0SboyrMM/edit#
>>
>> --
>> Best Regards,
>> Piotrek Koszuliński | CKEditor Lead Developer
>> --
>> CKSource – http://cksource.com | Follow CKSource on: Twitter
>> <http://twitter.com/cksource> | Facebook
>> <http://www.facebook.com/cksource> | Google+
>> <https://plus.google.com/+CKSource> | LinkedIn
>> <http://www.linkedin.com/company/cksource>
>>
>> On Wed, Feb 8, 2017 at 12:46 PM, Johannes Wilm <johannes@fiduswriter.org>
>> wrote:
>>
>>>
>>>
>>> On Wed, Feb 8, 2017 at 3:56 AM, Alexandre Elias <aelias@google.com>
>>> wrote:
>>>
>>>> OK, thanks for the clarifications.
>>>>
>>>> For me the bottom line I want to achieve is to have 1-to-1 mapping
>>>> between IME commands intended to mutate text and beforeinput+input pairs.
>>>> I don't like the idea of splitting any one of them into fragmentary pieces
>>>> nor of generating phantom ones not corresponding to any text mutation.
>>>> That generates unnecessary side effects and is difficult to reason about.
>>>>
>>>> Sometimes this would cause "natural" platform differences based on the
>>>> different behavior of the actual IMEs, and that's fine because that likely
>>>> means a different user-facing UX that JS would also need to conform to.  On
>>>> Android in the replacement scenario described, there is exactly one
>>>> mutation where the nodes are both collapsed and replaced by the new word,
>>>> so there should be exactly one input event.  On Mac OS X there is first a
>>>> visible collapse at the beginning of the composition, then (later on) a
>>>> change to the new word, so there should be exactly two input events.  I'm
>>>> okay with such "natural" complexity but not with artificial
>>>> spec-originating complexity.
>>>>
>>>> The idea of supporting a "preventDefault" on a fragmentary or phantom
>>>> event is particularly alarming to me, because I think "preventDefault"
>>>> should always be implemented by UAs as a simple early-return not performing
>>>> some action.
>>>>
>>> But the implication of "preventDefault" on a phantom event is that the
>>>> UA needs to proactively change the text when preventDefault is called but
>>>> do nothing when it isn't, which is weird and bound to lead to a cascade of
>>>> complexity.  In my way of seeing things, "cancelability" is not a sane
>>>> basis for fragmenting events, and the fragmenting is even counterproductive
>>>> to permitting workable cancelability.  (That's why I assumed the reason for
>>>> the split might be a "cannot-replace" assumption.)
>>>>
>>>
>>> That wasn't ever the intention. But course, having IME behave more
>>> similarly across browsers would be an advantage. It used to be the case
>>> that Chrome on desktop at the end of the composition would first remove the
>>> entire compositionstring and would then readd it. This was very convenient
>>> and it is why we made the last two events. From you description of it, you
>>> either no longer have those two things happening or you just don't have it
>>> on Android. That is less ideal.
>>>
>>> Ultimately I cannot see why IME input should not be standardized in some
>>> way when everything else is standardized.
>>>
>>>
>>> Imagine if instead of having keyboard, every country had inevented a
>>> diferent inut device. Most developers would likely support the devices of
>>> Germany, US, UK, France. But the input devices of Denmark and Nicaragua
>>> would have more difficulty finding traction. Some Asian countries that
>>> mainly use IME have a lot of population and Android is more popular than
>>> many desktop operating systems, but a lot of richtext editing is mainly
>>> confined to desktop (mobile clients often just have a function to write
>>> two-three words without styling which is realistic for someone on a metro
>>> to be doing) and a lot of things are mainly done in the European languages.
>>>
>>>
>>> For example, we may think that everyone should be able to write a W3C
>>> spec on their mobile phone in any possible langage, but in practice it will
>>> almost always be on done on dekstop and it will almost always be with
>>> keyboard input. In order to get support for the 1% of writers who will use
>>> IME anyway, it would be of great help for the IME in those 1% to behave the
>>> same way and not have 100 different IMEs with different behavor as that
>>> will mean that the JS developer will have to spend much more time than what
>>> is reasonable on something that is ultimately doesn't make any economic
>>> sense.
>>>
>>> So instead of dealing with all this complexity and hundreds of different
>>> ways of doing it, JS editors often decide that it's "good enough" to just
>>> diff the DOM before and after and then have a rough idea of what the user
>>> was trying to do.
>>>
>>>
>>>>
>>>> >  I don't quite get what you mean by combining steps two and three.
>>>> Step two was the reinsertion of the initial composition string into the
>>>> DOM. Step three is the change during the composition. There will usually be
>>>> many step threes, but only one step two.
>>>>
>>>> I simply meant combining step 2 into the first of the step 3s.
>>>>
>>>> > There is one problem in that compositionend with data="" could mean
>>>> the composition has been cancelled or that the final compositionstring is
>>>> empty.
>>>>
>>>> I propose to solve it always sending another step-3-style
>>>> insertCompositionText("") before the compositionEnd in the case that the
>>>> text is cleared, and no input event in the cancellation case.  Then there
>>>> would be no need to ever look at the data field of compositionEnd.
>>>>
>>>
>>> If it doesn't read the data attribute of compositionEnd, then the JS
>>> will have to follow all the compositionupdate events to figure out what the
>>> final string will be, right? And as long as the beforeinput events related
>>> to IME cannot be canceled and one has to wait for the entire IME string to
>>> be done, the JS will have to calculate and combine what the total
>>> targetrange of the composition is, right? If we don't really give the JS
>>> developers anythign new with the beofreinput event, why would it be used?
>>> Why wouldn't they just continue to diff the relevant part of the DOM?
>>>
>>>
>>>>
>>>> > Btw -- just to make it clear because this seems to have been a
>>>> common misconception between JS developers and browser developers in the
>>>> past: When we ask for these things to be cancelable, in 80% of cases this
>>>> is not because we want to stop the user from doing things. It's more than
>>>> we want to do the corresponding DOM changes in JS code, because they are
>>>> specific to the JS editor the user is currently using and they may not
>>>> necessarily make sense to the browser.
>>>>
>>>> That's useful context.  But my take from this is that beforeinput
>>>> preventDefaulting in *any* form cannot be used to meet this use case, at
>>>> least on Android.  Reason: on Android, we need to call the IME with
>>>> "restartInput()" whenever the text changes unexpectedly, which fully
>>>> reinitializes the IME and cancels ongoing compositions.  This call is not
>>>> at all desirable or needed if the JS editor is doing essentially the same
>>>> thing as the default implementation would have.  But unfortunately, there
>>>> is no sane way for UAs to know whether the JS is "trying" to stop the
>>>> action or if it's "trying" to recreate it.  We'd have to bias towards
>>>> oversending or undersending restartInput, likely via unreliable
>>>> heuristics.  The bugginess would be severe with either kind of bias.
>>>>
>>>> An alternate proposal: what if, when JS modifies the DOM within the
>>>> scope of any "input" event, then the UA would run a before/after comparison
>>>> to verify whether the summation of JS changes wound up changing the
>>>> plain text distillation of the DOM (and therefore whether it's provable
>>>> that the IME can sit blissfully unaware that anything strange happened).
>>>> If the plain text representation changed, then the UA would always send
>>>> restartInput and cancel composition, as UAs largely do in response to
>>>> unexpected JS changes today.  But if the only thing that changed was
>>>> styling or other rich DOM substructure, the UA would *not* send
>>>> restartInput and it would also reconstruct the composition range to keep it
>>>> at the same plain-text-character offsets against the new DOM nodes.
>>>>
>>>
>>>
>>> Ok, this would kind of be to babysit the JavaScript, right? The problem
>>> is that the plaintext distillation may eb slightly different, if for
>>> example, the JavaScript uses animated gifs for some of the letters, or if
>>> it needs to insert an empty space node. So I would prefer to say that all
>>> preventdefualting of beforeinput will always jsut prevent the DOM update
>>> from going through, but not for the action behdind it to be cancelled (we
>>> did discuss at an earlier meeting if it was possible to make two types of
>>> preventdefault to be able to distinguish between them). But yes, your
>>> proposal does sound better than what we had until now and should work under
>>> almost all cricumstances (editors that insert animated gifs just have to
>>> add those when the composition is done).
>>>
>>>
>>>>
>>>> In this model, the JS editor would need to stash a copy of the previous
>>>> DOM at time of "beforeinput", allow the "input" to happen, and then
>>>> reconstruct the rich structure.  Then we could avoid the entire can of
>>>> worms of what particular IME events are "cancelable" or not and how to work
>>>> around cases where they are not.
>>>>
>>>
>>> I don't understand. Keeping the DOM at the start of the composition and
>>> diffing when it's done, etc. is what is happening already. The beforeinput
>>> event seems to not add much value for that.
>>>
>>> I thought you had just made a suggestion on how to make beforeinput
>>> events during IME cancelable.
>>>
>>> To sum up, these two would give an advantage over todays situation:
>>>
>>> 1. Have all five beforeinput events, of which the middle three  are not
>>> cancelable and the first two only happen if one is recomposing. The 1st and
>>> the last of these events can be used by JS to adjust the IME so that it
>>> happens where it should and the final result is placed according to its own
>>> logic.
>>>
>>> 2. Have one, two or three different input events that are cancelablein
>>> so that the JS can manage the DOM updates itself.
>>>
>>> What gives hardly any advantage over today's situation:
>>>
>>> * Have one or two noncancelable beforeinput events that are not
>>> surrounded by cancelable ones.
>>>
>>> What cripples the beforeinput event for most use cases even on desktop:
>>>
>>> * Make most beforeinput events non-cancelable.
>>>
>>>
>>>
>>> The only advantage left of having the beforeinput event that I can see
>>> is:
>>>
>>> * handling UA button clicks and keyboard shortcuts
>>>
>>> * I understand that some editors don't monitor how the selection
>>> changes, so they may not have known where the selection was before
>>> something happened when it is done.
>>>
>>>
>>> The handling of UA native buttons is further complicated by the fact
>>> that browsers have a concept of a global undo stack, whereas all JS editors
>>> we have been able to find use a undo stack that is specific to one
>>> contenteditable host. As long as there is just one editing host on a page
>>> or one encloses all editing hosts in iframes, this doesn't seem to be a
>>> problem, but in all other cases it will mean that under some circumstances
>>> the undo and redo buttons will not behave as expected. From Safri we even
>>> heard they will not be able to turn these buttons on and off as needed. So
>>> what we will be shipping will nor work correctly for history in Safari
>>> right from the start.
>>>
>>> Which leaves us with buttons for richtext operations such as "make bold"
>>> and "make italic". As far as I know, these are only shipped with Safari, so
>>> really only Safari would need to add these.
>>>
>>>
>>>>
>>>> --
>>>> Alex
>>>>
>>>> On Mon, Feb 6, 2017 at 7:44 PM, Johannes Wilm <johannes@fiduswriter.org
>>>> > wrote:
>>>>
>>>>>
>>>>>
>>>>> On Tue, Feb 7, 2017 at 4:00 AM, Alexandre Elias <aelias@google.com>
>>>>> wrote:
>>>>>
>>>>>> On Mon, Feb 6, 2017 at 6:18 PM, Johannes Wilm <
>>>>>> johannes@fiduswriter.org> wrote:
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Feb 7, 2017 at 12:23 AM, Alexandre Elias <aelias@google.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi, I'm the maintainer of IME on Android.  I'll be happy to attend
>>>>>>>> the conference call next week.  As you've surmised, Android does need to
>>>>>>>> perform a DOM change to merge two nodes "later".  However, I disagree that
>>>>>>>> those 5 steps are somehow needed.  Our plan is to send a single
>>>>>>>> insertCompositionText beforeinput event instead of 5 events.
>>>>>>>>
>>>>>>>
>>>>>>> Nice to meet you Alexandre!
>>>>>>>
>>>>>>> It's a bit unfrotunate that you haven't been able to participate in
>>>>>>> the last 3 years of discussions about this subject (I think some people
>>>>>>> have even been discussing this for more than 8 years), and it seems that
>>>>>>> the mailing list isn't as alive any more as it once used to be, but I'll do
>>>>>>> my best to give the reasoning why we came to the conclusion that these
>>>>>>> events are needed. I would also recommend consulting the various open and
>>>>>>> closed issues on github and to consult the archive of the mailing list.
>>>>>>> Also, various Google people have been behind a lot if not most of the
>>>>>>> various various concepts, so they can probably give you more background
>>>>>>> about this as well.
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> Arguments why each is not needed:
>>>>>>>>
>>>>>>>>  > 1. deleteByComposition --This step is only needed when
>>>>>>>> recomposing. A composed word can go across element boundaries, so this may
>>>>>>>> delete more than just contents of a text node. (fx " He<b>ll</b><i>o
>>>>>>>> fish</i>!"). This step needs to be executed at the moment your Android IME
>>>>>>>> changes the DOM, whether this is at the moment the user taps the word and
>>>>>>>> the underline appears or at a later stage.
>>>>>>>>
>>>>>>>> There's no need for a separate "delete" event, because
>>>>>>>> insertion-to-replace-range is general enough to represent a deletion range
>>>>>>>> as well.  I prefer to reserve "delete" events exclusively for intended IME
>>>>>>>> deletions (e.g. select word and press backspace).
>>>>>>>>
>>>>>>>
>>>>>>> But I am a little confused: From the proposal Chong sent in, it
>>>>>>> sounded like you just wouldn't have any beforeinput event for this initial
>>>>>>> deletion. That would create a range of different problems, such as:
>>>>>>>
>>>>>>
>>>>>> I agree we should not hide any editing information from JS.  I am
>>>>>> saying that event 2/3 can encompass the deletion.  An implied, wrong
>>>>>> assumption underlying your argument seems to be that an insertion operation
>>>>>> can only ever add new text but not remove existing text.
>>>>>>
>>>>>
>>>>>
>>>>> No, that must be a misunderstanding. I don't know why you would think
>>>>> that. As I said previsouly, if  beforeinput events during IME are
>>>>> cancelable, 1 (deletion) and 2 (addition cshould be combinable into one
>>>>> single event,
>>>>>
>>>>>
>>>>> I don't quite get what you mean by combining steps two and three. Step
>>>>> two was the reinsertion of the initial composition string into the DOM.
>>>>> Step three is the change during the composition. There will usually be many
>>>>> step threes, but only one step two.
>>>>>
>>>>> The reason there is a step one and two at all is that the IME converts
>>>>> the composition string from being something that can contain a complex
>>>>> html-structure to just be  a plaintext string (in most cases anyway).
>>>>>
>>>>> So step 1 will remove "He<b>l</b>p" and step 2 will re-add it as
>>>>> "Help". Combining step 1 + 2 should work if step 2 is also cancelable.
>>>>> Combining step 2 and 3, or maybe 1 and 2 and 3 (?) would mean that we go
>>>>> straight from "He<b>l</b>p" to "Hel". and then the next two step 3 would
>>>>> add "l" and "o". In many cases the JS will probably be able to guess what
>>>>> the individual parts of this combination of 1/2/3 initially were (what part
>>>>> was just about turning it into a plaintext and which part was then about
>>>>> modifying the plaintext), but it would seem a bit meshed and I'm not sure
>>>>> one always can determine it. Nevertheless, if iME input is cancelable, then
>>>>> this does sound better than what was on the table previosuly.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>>   In reality, insertions are always *replacements* of the content of
>>>>>> the caret/selection/composition with text (for example, if you select text
>>>>>> with the mouse and then type "a" on a physical keyboard, no deletion event
>>>>>> is sent).
>>>>>>
>>>>>
>>>>> right. That'sw hy we have targetranges. But the difference for
>>>>> keyboard inout is that it is cancelable and there is no automatic
>>>>> normalization to plaintext. This is different than the situation with IME.
>>>>>
>>>>>>
>>>>>>
>>>>>>> * The more advanced editors today are moving toward a model where
>>>>>>> they hold a json version of the document which is separate from the DOM.
>>>>>>> They only make adjustments to the DOM when needed, and if the user changes
>>>>>>> something about the DOM, they make adjustments to the  json document. The
>>>>>>> json version is the authoritative copy of the document. For this to work,
>>>>>>> they need to know about any change to the DOM. If you make user-initiated
>>>>>>> DOM changes that are not causing any beforeinput event at all, the
>>>>>>> beforeinput event is somewhat useless, aand they will have to continue to
>>>>>>> monitor user initiated DOM-changes in other ways.
>>>>>>>
>>>>>>> * In a collaborative editor, if you make a change to the DOM of one
>>>>>>> user, and the JS is not being made aware of this change, then it won't
>>>>>>> notify the collaborators of this change, and so the collaborator will ahve
>>>>>>> different versions of the document. Again, the point of beforeinput events
>>>>>>> will be gone. JS develoeprs will use mutation observers or some other
>>>>>>> technology instead.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> > 2. insertCompositionText -- Also this step only needs to happen
>>>>>>>> at the moment only for recompositions. It reinserts the word from step one,
>>>>>>>> but this time as plaintext in one element (fx "Hello" => "<i>Hello
>>>>>>>> fish</i>")
>>>>>>>>
>>>>>>>> On Android because of the "late" timing, this merge operation would
>>>>>>>> happen in the same message loop task as step 3, so there is no need for
>>>>>>>> this event.  We prefer to go directly from "He<b>ll</b><i>o fish</i>" ->
>>>>>>>> "<i>Help<i>".
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Yes, in a perfect world, where all IME-related events are
>>>>>>> cancelable, this could be done and would be preferable, as for example
>>>>>>> Yosin has pointed out ( https://github.com/w3c/input-e
>>>>>>> vents/issues/45#issuecomment-261427648 ). We had a long discussion
>>>>>>> during the fall of 2015 about this, and among other ones several developers
>>>>>>> from Chromium eventually convinced us here that IME-events cannot be
>>>>>>> guaranteed to be cancelable. Even moving the caret to somewhere else during
>>>>>>> the composition may somehow crash the IME (hwo this is possible is a bit of
>>>>>>> a mystery to me as a JS developer).
>>>>>>>
>>>>>>> One could then say ok, combine 1 and 2 and make them both
>>>>>>> non-cancelable. The problem with that is that the browser doesn't always
>>>>>>> know the seamntic meaning of the various DOM elements used within a
>>>>>>> document and it may end up making decisions of what to delete that are
>>>>>>> problematic.
>>>>>>>
>>>>>>
>>>>>> I've been trying to figure out where these claims about IME from some
>>>>>> Chromium team members came from exactly and haven't figured it out, nobody
>>>>>> has a precise memory.  They seem to be based on a vague fears rather than
>>>>>> concrete knowledge of how IMEs behave.  I agree with your instinct that the
>>>>>> assumptions behind it don't make too much sense, that's why I think we
>>>>>> should hold off before we commit to a complicated model based on those
>>>>>> assumptions.
>>>>>>
>>>>>
>>>>> Well, maybe you guys need to discuss that internally. Until now what
>>>>> we said here is that we will assume the worst case scenario (that IMEs not
>>>>> being allowed to directly modify the DOM will lead to them crashing) and
>>>>> then in a future version to hopefully get one that doesn't crash.
>>>>>
>>>>> Btw -- just to make it clear because this seems to have been a common
>>>>> misconception between JS developers and browser developers in the past:
>>>>> When we ask for these things to be cancelable, in 80% of cases this is not
>>>>> because we want to stop the user from doing things. It's more than we want
>>>>> to do the corresponding DOM changes in JS code, because they are specific
>>>>> to the JS editor the user is currently using and they may not necessarily
>>>>> make sense to the browser.
>>>>>
>>>>>
>>>>>
>>>>>>
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> However, rniwa@apple.com has told us that on OS X, the standard
>>>>>>>> platform behavior is to merge the nodes at time of composition start.
>>>>>>>> Therefore, on OS X step 2 is still needed and there would be 2 beforeinput
>>>>>>>> events in total, but on Android there would be only 1.  This represents a
>>>>>>>> substantive platform difference that I think it makes sense to allow to
>>>>>>>> differ in the spec.  I don't think we should try to artificially
>>>>>>>> synchronize the platforms by always sending 2 events -- on Android this
>>>>>>>> would simply result in unnecessary work and side effects that would never
>>>>>>>> be seen by the user.
>>>>>>>>
>>>>>>>
>>>>>>> I think you should instead consider making this two events on
>>>>>>> Android. Android is really great, but text editing in Chrome on Android is
>>>>>>> really problematic especially when it comes to IME recomposition, as I am
>>>>>>> sure you are aware. I have tried various keyboard, and but even on
>>>>>>> mainstream sites such as Facebook, using a Google-provided keyboard,
>>>>>>> recomposition tends to just mess up entire sentences, throw random words
>>>>>>> around, deleting or merging words that are close to what one is trying to
>>>>>>> recompose, etc.
>>>>>>>
>>>>>>
>>>>>> Right, we are in process of fixing all of these bugs and expect to
>>>>>> get the remaining ones fixed in M58/M59 timeframe.  Anyway, this doesn't
>>>>>> address my point.  Personally, I believe that firing additional unnecessary
>>>>>> events will prove harmful to both UAs and JS editors -- in reality an
>>>>>> atomic IME change that just happened, but you are proposing to drip-feed it
>>>>>> into many separate events.  Your proposal doesn't provide JS any additional
>>>>>> information (it just splits information into pieces), so therefore I don't
>>>>>> think it can possibly be needed for any use case either.
>>>>>>
>>>>>>
>>>>>>
>>>>>>> Speed is not as much a concern anyway because it all depends on the
>>>>>>> human who is interacting with it, but in addition these really major issues
>>>>>>> mean that recomposition as it exists on Android as of today is not very
>>>>>>> useful. In addition there are the various issues that JS developers have
>>>>>>> complained about for a while: no compositionstart event, etc. . I am sure
>>>>>>> you know the list of grievances with it all too well.
>>>>>>>
>>>>>>
>>>>>> We should always be firing compositionStart today.  Feel free to file
>>>>>> a bug against me if you know of a case where we don't.
>>>>>>
>>>>>
>>>>> I recently sent a list of prioritized bugs from several JS editor
>>>>> projects to the Chromium team that they had requested from em. I assume
>>>>> they forwarded those issues relevant to Android to you.
>>>>>
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> Not knowing the details, but I could imagine you can check for event
>>>>>>> listeners, and only if those are present you make it two events.
>>>>>>>
>>>>>>> But as an alternative, and to try to get a bit closer to both your
>>>>>>> and Yosin's position, how about this:  We make two IME models, one for IMEs
>>>>>>> where the beforeinput events can be cancelled in which step 1 and 2 are
>>>>>>> merged and one where they cannot be cancelled where they are separate.
>>>>>>>
>>>>>>
>>>>>> Disagree, we should keep things as simple as possible.  I want us to
>>>>>> fire a minimum of events and avoid complicated logic forking.  I don't want
>>>>>> us to introduce permanent complexity into the platform for the sake of a
>>>>>> short-term compromise.
>>>>>>
>>>>>
>>>>> Well, you said that Apple does one thing and you do another. So either
>>>>> the two of you can agree or you don't. If you cannot agree, we'll have to
>>>>> have two different models.
>>>>>
>>>>> In theis scenario, if the Chrome model consists of just cutting out 4
>>>>> of 5 events on Android, then as I tried to explain in the last few emails
>>>>> and the comments on the proposal doc, the event will no longer function to
>>>>> build an editor on top of. At least not without combining it with mutation
>>>>> observers and other technologies.
>>>>>
>>>>> Having a model where IME events are cancelable is really something we
>>>>> all hoped to do in a future version of the spec. If the situation is such
>>>>> that Chromium/Android would prefer to do this right now, whereas Apple
>>>>> prefers to stay with the non-cancelable IME-model then that would not seem
>>>>> like terribly different models.
>>>>>
>>>>>
>>>>>>
>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>> > 4. deleteCompositionText -- removed (fx "help"): It removed the
>>>>>>>> text from the underline and it may not be the entire composition.
>>>>>>>>
>>>>>>>> > 5. insertFromComposition -- adds "help" permanently, and gives
>>>>>>>> the JS the opposrtunity to rebuild the complex HTML structure lost in step
>>>>>>>> 1.
>>>>>>>>
>>>>>>>> Step 3 is already permanent.  The fact that there is still an
>>>>>>>> active composition underline at the time of step 3 does not make it somehow
>>>>>>>> temporary.  It has already been injected into the DOM and caused all side
>>>>>>>> effects.  So the JS should rebuild its complex structure at time of event
>>>>>>>> 3, and events 4/5 are redundant.
>>>>>>>>
>>>>>>>
>>>>>>> When a composition starts, they temporarily pause most activities
>>>>>>> because they are not really allowed to intervene. Then, once the
>>>>>>> composition is finished, they deal with the finished text.  input. For
>>>>>>> example, in step 5 they may reapply some partial styling that was lost in
>>>>>>> step 1.
>>>>>>>
>>>>>>
>>>>>> "compositionend" event is fired at the same timing that steps 4/5
>>>>>> would be, isn't it sufficient for this use case?
>>>>>>
>>>>>
>>>>> There is one problem in that compositionend with data="" could mean
>>>>> the composition has been cancelled or that the final compositionstring is
>>>>> empty. The distinction between 4 and 5 is needed if 2, 3 & 4 are
>>>>> non-cancelable. If they are cancelable, then one should be able to combine
>>>>> 1 and 2, and skip 4 and 5 and the JS would continously make any adjustments
>>>>> it needs to the DOM (while taking into consideration what the
>>>>> compositionstring is at any moment).
>>>>>
>>>>>
>>>>>>
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>> The reason you state for the proposed change is that "On Android
>>>>>>>>> it’s very common for IME to start composition whenever a cursor is moved
>>>>>>>>> onto a word.  This is primarily intended to passively show an underline to
>>>>>>>>> the user (to indicate what would be replaced *if* user selects an alternate
>>>>>>>>> spelling)."
>>>>>>>>>
>>>>>>>>> If such tapping leads to actual DOM changes, then  the JS needs to
>>>>>>>>> know about those(step 1/2). But if you don't change the DOM immediately and
>>>>>>>>> only do so at a later stage when you "really" start the composition, then
>>>>>>>>> you can also wait until that stage.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> So these steps are needed on allplattforms, no matter whether or
>>>>>>>>> not you change how IMEs work on Android.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Adding aelias@ to make sure he is able to make the call on the
>>>>>>>>>> 14th.
>>>>>>>>>>
>>>>>>>>>> dave.
>>>>>>>>>>
>>>>>>>>>> On Fri, Feb 3, 2017 at 1:33 AM, Johannes Wilm <
>>>>>>>>>> johannes@fiduswriter.org> wrote:
>>>>>>>>>>
>>>>>>>>>>> Could we get the person or people who came up with these changes
>>>>>>>>>>> due to Android to participate in the call on the 14th? I think it's very
>>>>>>>>>>> important to have them invovled in this discussion.
>>>>>>>>>>>
>>>>>>>>>>> On Fri, Feb 3, 2017 at 10:21 AM, Johannes Wilm <
>>>>>>>>>>> johannes@fiduswriter.org> wrote:
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Fri, Feb 3, 2017 at 10:18 AM, Johannes Wilm <
>>>>>>>>>>>> johannes@fiduswriter.org> wrote:
>>>>>>>>>>>> ...
>>>>>>>>>>>>
>>>>>>>>>>>>> Right now I cannot see any purpose of the beforeinput event
>>>>>>>>>>>>> with these changes applied, but maybe I am missing something?
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>> I need to restarct that. I guess it could still be useful to
>>>>>>>>>>>> stop native bold/italic buttons from making their own, non-controlled DOM
>>>>>>>>>>>> changes. It's just for text input that it is irrelevant. And IME will
>>>>>>>>>>>> continue to be a mess.
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> Johannes Wilm
>>>>>>>>>>>> Fidus Writer
>>>>>>>>>>>> http://www.fiduswriter.org
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Johannes Wilm
>>>>>>>>>>> Fidus Writer
>>>>>>>>>>> http://www.fiduswriter.org
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Johannes Wilm
>>>>>>>>> Fidus Writer
>>>>>>>>> http://www.fiduswriter.org
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Johannes Wilm
>>>>>>> Fidus Writer
>>>>>>> http://www.fiduswriter.org
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Johannes Wilm
>>>>> Fidus Writer
>>>>> http://www.fiduswriter.org
>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Johannes Wilm
>>> Fidus Writer
>>> http://www.fiduswriter.org
>>>
>>
>>
>
>
> --
> Johannes Wilm
> Fidus Writer
> http://www.fiduswriter.org
>



-- 
Johannes Wilm
Fidus Writer
http://www.fiduswriter.org