Re: 2 Proposals for Minimum Viable InputEvent

On Wed, Feb 8, 2017 at 3:59 PM, Piotr Koszuliński <
p.koszulinski@cksource.com> wrote:

> Hey,
>
> I won't try to go too deep into how IME related events should work because
> I don't know IME mechanisms and all its variations well enough, despite
> working with cE for more than 5 years. Perhaps, this answers some questions
> like how hard IME is actually and what's the average knowledge about it,
> even amongst quite experienced RTE developers ;).
>
> Anyway, I'd like to rather focus on how non-preventable the
> beforeinput event can be to make any value for CKEditor 5. I hope it will
> provide some context for the discussion.
>
>
> ### Text insertion ############
>
> A couple of months ago I created a PoC of CKEditor 5's typing feature
> working based on the beforeinput event available at that time in Blink [1].
> Unfortunately, I can't quickly check how the editor would behave if the
> event wasn't cancelled, but that would be very similar to how the editor
> needs to work without the beforeinput event.
>
> So how it works now?
>
> 1. We use a mutation observer to catch the input (typed characters).
> There's a special case when selection isn't empty when the user starts to
> type. We needed to hardcode the unsafe keys and delete the selected content
> on keydown. Otherwise, typing a single letter might cause hundreds of
> mutations if selection crossed some complicated content. We must avoid that.
> 2. We diff the DOM with our virtual DOM implementation to get a simplified
> form of the mutations. Browser may fire dozens of them, while for us it can
> be just a few changed text nodes or  removed elements.
> 3. Then we analyse the mutations. We try to find a fragment of removed and
> inserted text, plus, the position of the selection after the change.
> 4. Then, if a feature (in this case, the typing feature) can handle
> certain mutations (i.e. they were not too complex) it applies the necessary
> changes to the model.
> 5. The changes in the model are converted to the virtual DOM.
> 6. If virtual DOM differs from the real DOM, minimal changes are applied.
>
> This architecture allows us to:
>
> * Apply understandable changes to the model, without re-rendering the DOM.
> This is crucial to support IME because some changes in the DOM break the
> composition (see the "IME" section in [1]).
> * Re-render the DOM before paint happens if changes were rejected by the
> editor. That, fortunately, needs to happen very rarely, but it's nice that
> there's no flickering anyway.
>
> How would beforeinput simplify our implementation?
>
> * We wouldn't need to listen to mutations and normalize them (1. & 2.).
> * We wouldn't need that ugly unsafe keys handler meant to empty the
> selection before a character is inserted (1.).
> * We would avoid the diffing mechanism, which is quite complex already
> despite missing few edge cases with spell checking or auto correction (3.).
> * We would have a perfect clarity whether the inserted space was meant to
> be a space or non-breaking space (it's problem to learn that when it's at
> the end of a block).
> * Etc, etc.
>


Shouldn't you have the same information iof you listened to all the
compositionupdate events (and these events actually delivered accurate
data)?


>
> So, that's the profit even if we can't cancel the event. It's still
> a pretty significant simplification, comparing to what we have to do now.
>
> However, there's one nasty case which may still require hacking if the
> event isn't cancelled. It's how the editor can delete the content itself
> when the user is typing over a non-empty selection. We can't let the
> browser do that because we wouldn't know how to convert those changes to
> the model (and sometimes it might simply be impossible). In general, there
> are a couple of options here.
>
> * If we'd be able to cancel all beforeinput events, that would be perfect,
> because we'd be able to handle the deletion of the content by ourselves
> first, and then we'd insert the typed characters.
> * If the beforeinput event is not cancellable, then:
>    * it'd be nice if for the non-empty selection there would be
> a beforeinput with type=deleteContent fired before beforeinput with
> type=insert* (that would natively split the process into two steps –
> deletion and insertion),
>    * alternatively, we should be able to change the DOM
> on beforeinput event without breaking the native insertion itself (then we
> can simply delete the selected content and set selection in a correct
> place, where the new character should be inserted).
>
> One of these options must be true because, otherwise, we won't be able to
> use the event. We'd still need to base on some ugly heuristic for
> discovering keys which are about to delete the content of the selection.
>
> What I don't know is how these options apply to IME. Let's assume that its
> events are non-cancellable, so we get some beforeinput event of delete*
> type followed by insertCompositionText. The deletion part is unclear for me
> because the insertion might've been in a middle of composition (so you get
> deleteCompositionText + insertCompositionText) or it might be the first
> composed character. What do we get in this case?
>
> I'm asking because I assume that when the user is composing already, the
> browser will never do some significant DOM changes. Composition would
> normally only touch the text nodes in a certain block, but not the block
> itself. Besides, while composition takes place, we're not able to re-render
> the DOM anyway, because that breaks the composition (see the "IME section
> in [1]). So, if something is wrong, we need to wait for the composition to
> finish anyway. But this is acceptable because I don't expect significant
> changes to be required. Maybe just some inline styling.
>
> However, if the insertion starts the composition, then the content which
> was previously selected might contain multiple blocks, tables, etc. and we
> must handle deletion of that piece ourselves. So, at least in this case, we
> must be able to either act upon inserComopositionText or there must be a
> separate deleteContent event.
>
> Uhh... that was about text insertion.
>
>
> ### Other beforeinput types ##################
>
> A non-cancellable beforeinput for the text insertion (and text deletion)
> is acceptable (with couple limitations, as I tried to explain above).
> However, other before input events must be cancellable. Otherwise, there
> will be quite pointless.
>
> For example, how do we handle the backspace and delete keys now? We listen
> to keydown, discover which key was pressed and with what modifiers and
> delete the content. We might not be able to implement "delete word" or
> "delete soft line" very precisely, but these are really minor issues. So,
> in this case the before input wouldn't give us anything. Actually, it'd be
> even worse than listening to the keydown event because we can cancel the
> keydown event.
>
> The same with text formatting – we can easily listen to CTRL+B to know
> whether someone wants to bold a text. We can cancel the keydown event. It'd
> be much worse if we had to listen to the beforeinput event and fix the DOM
> broken by the browser which incorrectly applied some <b> tag. The
> beforeinput event would only help with localizing the keystroke (in some
> languages it won't be CTRL+B).
>
> And so on, and so on.
>
> I can understand that IME makes some beforeinput events non-cancellable.
> But I totally don't understand why would deleteContentForward or
> insertUndeorderList be non-cancellable too (as was proposed in the document
> [2]). The set of non-cancellable events should be minimal. Otherwise, the
> situation is not going to improve.
>
>
>
> [1] https://github.com/w3c/editing/issues/149
> [2] https://docs.google.com/document/d/1yPZEkHl_WOPjVeilZjE1XmlyjLCe-
> uS7THI0SboyrMM/edit#
>
> --
> Best Regards,
> Piotrek Koszuliński | CKEditor Lead Developer
> --
> CKSource – http://cksource.com | Follow CKSource on: Twitter
> <http://twitter.com/cksource> | Facebook
> <http://www.facebook.com/cksource> | Google+
> <https://plus.google.com/+CKSource> | LinkedIn
> <http://www.linkedin.com/company/cksource>
>
> On Wed, Feb 8, 2017 at 12:46 PM, Johannes Wilm <johannes@fiduswriter.org>
> wrote:
>
>>
>>
>> On Wed, Feb 8, 2017 at 3:56 AM, Alexandre Elias <aelias@google.com>
>> wrote:
>>
>>> OK, thanks for the clarifications.
>>>
>>> For me the bottom line I want to achieve is to have 1-to-1 mapping
>>> between IME commands intended to mutate text and beforeinput+input pairs.
>>> I don't like the idea of splitting any one of them into fragmentary pieces
>>> nor of generating phantom ones not corresponding to any text mutation.
>>> That generates unnecessary side effects and is difficult to reason about.
>>>
>>> Sometimes this would cause "natural" platform differences based on the
>>> different behavior of the actual IMEs, and that's fine because that likely
>>> means a different user-facing UX that JS would also need to conform to.  On
>>> Android in the replacement scenario described, there is exactly one
>>> mutation where the nodes are both collapsed and replaced by the new word,
>>> so there should be exactly one input event.  On Mac OS X there is first a
>>> visible collapse at the beginning of the composition, then (later on) a
>>> change to the new word, so there should be exactly two input events.  I'm
>>> okay with such "natural" complexity but not with artificial
>>> spec-originating complexity.
>>>
>>> The idea of supporting a "preventDefault" on a fragmentary or phantom
>>> event is particularly alarming to me, because I think "preventDefault"
>>> should always be implemented by UAs as a simple early-return not performing
>>> some action.
>>>
>> But the implication of "preventDefault" on a phantom event is that the UA
>>> needs to proactively change the text when preventDefault is called but do
>>> nothing when it isn't, which is weird and bound to lead to a cascade of
>>> complexity.  In my way of seeing things, "cancelability" is not a sane
>>> basis for fragmenting events, and the fragmenting is even counterproductive
>>> to permitting workable cancelability.  (That's why I assumed the reason for
>>> the split might be a "cannot-replace" assumption.)
>>>
>>
>> That wasn't ever the intention. But course, having IME behave more
>> similarly across browsers would be an advantage. It used to be the case
>> that Chrome on desktop at the end of the composition would first remove the
>> entire compositionstring and would then readd it. This was very convenient
>> and it is why we made the last two events. From you description of it, you
>> either no longer have those two things happening or you just don't have it
>> on Android. That is less ideal.
>>
>> Ultimately I cannot see why IME input should not be standardized in some
>> way when everything else is standardized.
>>
>>
>> Imagine if instead of having keyboard, every country had inevented a
>> diferent inut device. Most developers would likely support the devices of
>> Germany, US, UK, France. But the input devices of Denmark and Nicaragua
>> would have more difficulty finding traction. Some Asian countries that
>> mainly use IME have a lot of population and Android is more popular than
>> many desktop operating systems, but a lot of richtext editing is mainly
>> confined to desktop (mobile clients often just have a function to write
>> two-three words without styling which is realistic for someone on a metro
>> to be doing) and a lot of things are mainly done in the European languages.
>>
>>
>> For example, we may think that everyone should be able to write a W3C
>> spec on their mobile phone in any possible langage, but in practice it will
>> almost always be on done on dekstop and it will almost always be with
>> keyboard input. In order to get support for the 1% of writers who will use
>> IME anyway, it would be of great help for the IME in those 1% to behave the
>> same way and not have 100 different IMEs with different behavor as that
>> will mean that the JS developer will have to spend much more time than what
>> is reasonable on something that is ultimately doesn't make any economic
>> sense.
>>
>> So instead of dealing with all this complexity and hundreds of different
>> ways of doing it, JS editors often decide that it's "good enough" to just
>> diff the DOM before and after and then have a rough idea of what the user
>> was trying to do.
>>
>>
>>>
>>> >  I don't quite get what you mean by combining steps two and three.
>>> Step two was the reinsertion of the initial composition string into the
>>> DOM. Step three is the change during the composition. There will usually be
>>> many step threes, but only one step two.
>>>
>>> I simply meant combining step 2 into the first of the step 3s.
>>>
>>> > There is one problem in that compositionend with data="" could mean
>>> the composition has been cancelled or that the final compositionstring is
>>> empty.
>>>
>>> I propose to solve it always sending another step-3-style
>>> insertCompositionText("") before the compositionEnd in the case that the
>>> text is cleared, and no input event in the cancellation case.  Then there
>>> would be no need to ever look at the data field of compositionEnd.
>>>
>>
>> If it doesn't read the data attribute of compositionEnd, then the JS will
>> have to follow all the compositionupdate events to figure out what the
>> final string will be, right? And as long as the beforeinput events related
>> to IME cannot be canceled and one has to wait for the entire IME string to
>> be done, the JS will have to calculate and combine what the total
>> targetrange of the composition is, right? If we don't really give the JS
>> developers anythign new with the beofreinput event, why would it be used?
>> Why wouldn't they just continue to diff the relevant part of the DOM?
>>
>>
>>>
>>> > Btw -- just to make it clear because this seems to have been a common
>>> misconception between JS developers and browser developers in the past:
>>> When we ask for these things to be cancelable, in 80% of cases this is not
>>> because we want to stop the user from doing things. It's more than we want
>>> to do the corresponding DOM changes in JS code, because they are specific
>>> to the JS editor the user is currently using and they may not necessarily
>>> make sense to the browser.
>>>
>>> That's useful context.  But my take from this is that beforeinput
>>> preventDefaulting in *any* form cannot be used to meet this use case, at
>>> least on Android.  Reason: on Android, we need to call the IME with
>>> "restartInput()" whenever the text changes unexpectedly, which fully
>>> reinitializes the IME and cancels ongoing compositions.  This call is not
>>> at all desirable or needed if the JS editor is doing essentially the same
>>> thing as the default implementation would have.  But unfortunately, there
>>> is no sane way for UAs to know whether the JS is "trying" to stop the
>>> action or if it's "trying" to recreate it.  We'd have to bias towards
>>> oversending or undersending restartInput, likely via unreliable
>>> heuristics.  The bugginess would be severe with either kind of bias.
>>>
>>> An alternate proposal: what if, when JS modifies the DOM within the
>>> scope of any "input" event, then the UA would run a before/after comparison
>>> to verify whether the summation of JS changes wound up changing the
>>> plain text distillation of the DOM (and therefore whether it's provable
>>> that the IME can sit blissfully unaware that anything strange happened).
>>> If the plain text representation changed, then the UA would always send
>>> restartInput and cancel composition, as UAs largely do in response to
>>> unexpected JS changes today.  But if the only thing that changed was
>>> styling or other rich DOM substructure, the UA would *not* send
>>> restartInput and it would also reconstruct the composition range to keep it
>>> at the same plain-text-character offsets against the new DOM nodes.
>>>
>>
>>
>> Ok, this would kind of be to babysit the JavaScript, right? The problem
>> is that the plaintext distillation may eb slightly different, if for
>> example, the JavaScript uses animated gifs for some of the letters, or if
>> it needs to insert an empty space node. So I would prefer to say that all
>> preventdefualting of beforeinput will always jsut prevent the DOM update
>> from going through, but not for the action behdind it to be cancelled (we
>> did discuss at an earlier meeting if it was possible to make two types of
>> preventdefault to be able to distinguish between them). But yes, your
>> proposal does sound better than what we had until now and should work under
>> almost all cricumstances (editors that insert animated gifs just have to
>> add those when the composition is done).
>>
>>
>>>
>>> In this model, the JS editor would need to stash a copy of the previous
>>> DOM at time of "beforeinput", allow the "input" to happen, and then
>>> reconstruct the rich structure.  Then we could avoid the entire can of
>>> worms of what particular IME events are "cancelable" or not and how to work
>>> around cases where they are not.
>>>
>>
>> I don't understand. Keeping the DOM at the start of the composition and
>> diffing when it's done, etc. is what is happening already. The beforeinput
>> event seems to not add much value for that.
>>
>> I thought you had just made a suggestion on how to make beforeinput
>> events during IME cancelable.
>>
>> To sum up, these two would give an advantage over todays situation:
>>
>> 1. Have all five beforeinput events, of which the middle three  are not
>> cancelable and the first two only happen if one is recomposing. The 1st and
>> the last of these events can be used by JS to adjust the IME so that it
>> happens where it should and the final result is placed according to its own
>> logic.
>>
>> 2. Have one, two or three different input events that are cancelablein so
>> that the JS can manage the DOM updates itself.
>>
>> What gives hardly any advantage over today's situation:
>>
>> * Have one or two noncancelable beforeinput events that are not
>> surrounded by cancelable ones.
>>
>> What cripples the beforeinput event for most use cases even on desktop:
>>
>> * Make most beforeinput events non-cancelable.
>>
>>
>>
>> The only advantage left of having the beforeinput event that I can see is:
>>
>> * handling UA button clicks and keyboard shortcuts
>>
>> * I understand that some editors don't monitor how the selection changes,
>> so they may not have known where the selection was before something
>> happened when it is done.
>>
>>
>> The handling of UA native buttons is further complicated by the fact that
>> browsers have a concept of a global undo stack, whereas all JS editors we
>> have been able to find use a undo stack that is specific to one
>> contenteditable host. As long as there is just one editing host on a page
>> or one encloses all editing hosts in iframes, this doesn't seem to be a
>> problem, but in all other cases it will mean that under some circumstances
>> the undo and redo buttons will not behave as expected. From Safri we even
>> heard they will not be able to turn these buttons on and off as needed. So
>> what we will be shipping will nor work correctly for history in Safari
>> right from the start.
>>
>> Which leaves us with buttons for richtext operations such as "make bold"
>> and "make italic". As far as I know, these are only shipped with Safari, so
>> really only Safari would need to add these.
>>
>>
>>>
>>> --
>>> Alex
>>>
>>> On Mon, Feb 6, 2017 at 7:44 PM, Johannes Wilm <johannes@fiduswriter.org>
>>> wrote:
>>>
>>>>
>>>>
>>>> On Tue, Feb 7, 2017 at 4:00 AM, Alexandre Elias <aelias@google.com>
>>>> wrote:
>>>>
>>>>> On Mon, Feb 6, 2017 at 6:18 PM, Johannes Wilm <
>>>>> johannes@fiduswriter.org> wrote:
>>>>>
>>>>>>
>>>>>>
>>>>>> On Tue, Feb 7, 2017 at 12:23 AM, Alexandre Elias <aelias@google.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi, I'm the maintainer of IME on Android.  I'll be happy to attend
>>>>>>> the conference call next week.  As you've surmised, Android does need to
>>>>>>> perform a DOM change to merge two nodes "later".  However, I disagree that
>>>>>>> those 5 steps are somehow needed.  Our plan is to send a single
>>>>>>> insertCompositionText beforeinput event instead of 5 events.
>>>>>>>
>>>>>>
>>>>>> Nice to meet you Alexandre!
>>>>>>
>>>>>> It's a bit unfrotunate that you haven't been able to participate in
>>>>>> the last 3 years of discussions about this subject (I think some people
>>>>>> have even been discussing this for more than 8 years), and it seems that
>>>>>> the mailing list isn't as alive any more as it once used to be, but I'll do
>>>>>> my best to give the reasoning why we came to the conclusion that these
>>>>>> events are needed. I would also recommend consulting the various open and
>>>>>> closed issues on github and to consult the archive of the mailing list.
>>>>>> Also, various Google people have been behind a lot if not most of the
>>>>>> various various concepts, so they can probably give you more background
>>>>>> about this as well.
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> Arguments why each is not needed:
>>>>>>>
>>>>>>>  > 1. deleteByComposition --This step is only needed when
>>>>>>> recomposing. A composed word can go across element boundaries, so this may
>>>>>>> delete more than just contents of a text node. (fx " He<b>ll</b><i>o
>>>>>>> fish</i>!"). This step needs to be executed at the moment your Android IME
>>>>>>> changes the DOM, whether this is at the moment the user taps the word and
>>>>>>> the underline appears or at a later stage.
>>>>>>>
>>>>>>> There's no need for a separate "delete" event, because
>>>>>>> insertion-to-replace-range is general enough to represent a deletion range
>>>>>>> as well.  I prefer to reserve "delete" events exclusively for intended IME
>>>>>>> deletions (e.g. select word and press backspace).
>>>>>>>
>>>>>>
>>>>>> But I am a little confused: From the proposal Chong sent in, it
>>>>>> sounded like you just wouldn't have any beforeinput event for this initial
>>>>>> deletion. That would create a range of different problems, such as:
>>>>>>
>>>>>
>>>>> I agree we should not hide any editing information from JS.  I am
>>>>> saying that event 2/3 can encompass the deletion.  An implied, wrong
>>>>> assumption underlying your argument seems to be that an insertion operation
>>>>> can only ever add new text but not remove existing text.
>>>>>
>>>>
>>>>
>>>> No, that must be a misunderstanding. I don't know why you would think
>>>> that. As I said previsouly, if  beforeinput events during IME are
>>>> cancelable, 1 (deletion) and 2 (addition cshould be combinable into one
>>>> single event,
>>>>
>>>>
>>>> I don't quite get what you mean by combining steps two and three. Step
>>>> two was the reinsertion of the initial composition string into the DOM.
>>>> Step three is the change during the composition. There will usually be many
>>>> step threes, but only one step two.
>>>>
>>>> The reason there is a step one and two at all is that the IME converts
>>>> the composition string from being something that can contain a complex
>>>> html-structure to just be  a plaintext string (in most cases anyway).
>>>>
>>>> So step 1 will remove "He<b>l</b>p" and step 2 will re-add it as
>>>> "Help". Combining step 1 + 2 should work if step 2 is also cancelable.
>>>> Combining step 2 and 3, or maybe 1 and 2 and 3 (?) would mean that we go
>>>> straight from "He<b>l</b>p" to "Hel". and then the next two step 3 would
>>>> add "l" and "o". In many cases the JS will probably be able to guess what
>>>> the individual parts of this combination of 1/2/3 initially were (what part
>>>> was just about turning it into a plaintext and which part was then about
>>>> modifying the plaintext), but it would seem a bit meshed and I'm not sure
>>>> one always can determine it. Nevertheless, if iME input is cancelable, then
>>>> this does sound better than what was on the table previosuly.
>>>>
>>>>
>>>>
>>>>
>>>>>   In reality, insertions are always *replacements* of the content of
>>>>> the caret/selection/composition with text (for example, if you select text
>>>>> with the mouse and then type "a" on a physical keyboard, no deletion event
>>>>> is sent).
>>>>>
>>>>
>>>> right. That'sw hy we have targetranges. But the difference for keyboard
>>>> inout is that it is cancelable and there is no automatic normalization to
>>>> plaintext. This is different than the situation with IME.
>>>>
>>>>>
>>>>>
>>>>>> * The more advanced editors today are moving toward a model where
>>>>>> they hold a json version of the document which is separate from the DOM.
>>>>>> They only make adjustments to the DOM when needed, and if the user changes
>>>>>> something about the DOM, they make adjustments to the  json document. The
>>>>>> json version is the authoritative copy of the document. For this to work,
>>>>>> they need to know about any change to the DOM. If you make user-initiated
>>>>>> DOM changes that are not causing any beforeinput event at all, the
>>>>>> beforeinput event is somewhat useless, aand they will have to continue to
>>>>>> monitor user initiated DOM-changes in other ways.
>>>>>>
>>>>>> * In a collaborative editor, if you make a change to the DOM of one
>>>>>> user, and the JS is not being made aware of this change, then it won't
>>>>>> notify the collaborators of this change, and so the collaborator will ahve
>>>>>> different versions of the document. Again, the point of beforeinput events
>>>>>> will be gone. JS develoeprs will use mutation observers or some other
>>>>>> technology instead.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> > 2. insertCompositionText -- Also this step only needs to happen at
>>>>>>> the moment only for recompositions. It reinserts the word from step one,
>>>>>>> but this time as plaintext in one element (fx "Hello" => "<i>Hello
>>>>>>> fish</i>")
>>>>>>>
>>>>>>> On Android because of the "late" timing, this merge operation would
>>>>>>> happen in the same message loop task as step 3, so there is no need for
>>>>>>> this event.  We prefer to go directly from "He<b>ll</b><i>o fish</i>" ->
>>>>>>> "<i>Help<i>".
>>>>>>>
>>>>>>
>>>>>>
>>>>>> Yes, in a perfect world, where all IME-related events are cancelable,
>>>>>> this could be done and would be preferable, as for example Yosin has
>>>>>> pointed out ( https://github.com/w3c/input-e
>>>>>> vents/issues/45#issuecomment-261427648 ). We had a long discussion
>>>>>> during the fall of 2015 about this, and among other ones several developers
>>>>>> from Chromium eventually convinced us here that IME-events cannot be
>>>>>> guaranteed to be cancelable. Even moving the caret to somewhere else during
>>>>>> the composition may somehow crash the IME (hwo this is possible is a bit of
>>>>>> a mystery to me as a JS developer).
>>>>>>
>>>>>> One could then say ok, combine 1 and 2 and make them both
>>>>>> non-cancelable. The problem with that is that the browser doesn't always
>>>>>> know the seamntic meaning of the various DOM elements used within a
>>>>>> document and it may end up making decisions of what to delete that are
>>>>>> problematic.
>>>>>>
>>>>>
>>>>> I've been trying to figure out where these claims about IME from some
>>>>> Chromium team members came from exactly and haven't figured it out, nobody
>>>>> has a precise memory.  They seem to be based on a vague fears rather than
>>>>> concrete knowledge of how IMEs behave.  I agree with your instinct that the
>>>>> assumptions behind it don't make too much sense, that's why I think we
>>>>> should hold off before we commit to a complicated model based on those
>>>>> assumptions.
>>>>>
>>>>
>>>> Well, maybe you guys need to discuss that internally. Until now what we
>>>> said here is that we will assume the worst case scenario (that IMEs not
>>>> being allowed to directly modify the DOM will lead to them crashing) and
>>>> then in a future version to hopefully get one that doesn't crash.
>>>>
>>>> Btw -- just to make it clear because this seems to have been a common
>>>> misconception between JS developers and browser developers in the past:
>>>> When we ask for these things to be cancelable, in 80% of cases this is not
>>>> because we want to stop the user from doing things. It's more than we want
>>>> to do the corresponding DOM changes in JS code, because they are specific
>>>> to the JS editor the user is currently using and they may not necessarily
>>>> make sense to the browser.
>>>>
>>>>
>>>>
>>>>>
>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> However, rniwa@apple.com has told us that on OS X, the standard
>>>>>>> platform behavior is to merge the nodes at time of composition start.
>>>>>>> Therefore, on OS X step 2 is still needed and there would be 2 beforeinput
>>>>>>> events in total, but on Android there would be only 1.  This represents a
>>>>>>> substantive platform difference that I think it makes sense to allow to
>>>>>>> differ in the spec.  I don't think we should try to artificially
>>>>>>> synchronize the platforms by always sending 2 events -- on Android this
>>>>>>> would simply result in unnecessary work and side effects that would never
>>>>>>> be seen by the user.
>>>>>>>
>>>>>>
>>>>>> I think you should instead consider making this two events on
>>>>>> Android. Android is really great, but text editing in Chrome on Android is
>>>>>> really problematic especially when it comes to IME recomposition, as I am
>>>>>> sure you are aware. I have tried various keyboard, and but even on
>>>>>> mainstream sites such as Facebook, using a Google-provided keyboard,
>>>>>> recomposition tends to just mess up entire sentences, throw random words
>>>>>> around, deleting or merging words that are close to what one is trying to
>>>>>> recompose, etc.
>>>>>>
>>>>>
>>>>> Right, we are in process of fixing all of these bugs and expect to get
>>>>> the remaining ones fixed in M58/M59 timeframe.  Anyway, this doesn't
>>>>> address my point.  Personally, I believe that firing additional unnecessary
>>>>> events will prove harmful to both UAs and JS editors -- in reality an
>>>>> atomic IME change that just happened, but you are proposing to drip-feed it
>>>>> into many separate events.  Your proposal doesn't provide JS any additional
>>>>> information (it just splits information into pieces), so therefore I don't
>>>>> think it can possibly be needed for any use case either.
>>>>>
>>>>>
>>>>>
>>>>>> Speed is not as much a concern anyway because it all depends on the
>>>>>> human who is interacting with it, but in addition these really major issues
>>>>>> mean that recomposition as it exists on Android as of today is not very
>>>>>> useful. In addition there are the various issues that JS developers have
>>>>>> complained about for a while: no compositionstart event, etc. . I am sure
>>>>>> you know the list of grievances with it all too well.
>>>>>>
>>>>>
>>>>> We should always be firing compositionStart today.  Feel free to file
>>>>> a bug against me if you know of a case where we don't.
>>>>>
>>>>
>>>> I recently sent a list of prioritized bugs from several JS editor
>>>> projects to the Chromium team that they had requested from em. I assume
>>>> they forwarded those issues relevant to Android to you.
>>>>
>>>>>
>>>>>
>>>>>>
>>>>>> Not knowing the details, but I could imagine you can check for event
>>>>>> listeners, and only if those are present you make it two events.
>>>>>>
>>>>>> But as an alternative, and to try to get a bit closer to both your
>>>>>> and Yosin's position, how about this:  We make two IME models, one for IMEs
>>>>>> where the beforeinput events can be cancelled in which step 1 and 2 are
>>>>>> merged and one where they cannot be cancelled where they are separate.
>>>>>>
>>>>>
>>>>> Disagree, we should keep things as simple as possible.  I want us to
>>>>> fire a minimum of events and avoid complicated logic forking.  I don't want
>>>>> us to introduce permanent complexity into the platform for the sake of a
>>>>> short-term compromise.
>>>>>
>>>>
>>>> Well, you said that Apple does one thing and you do another. So either
>>>> the two of you can agree or you don't. If you cannot agree, we'll have to
>>>> have two different models.
>>>>
>>>> In theis scenario, if the Chrome model consists of just cutting out 4
>>>> of 5 events on Android, then as I tried to explain in the last few emails
>>>> and the comments on the proposal doc, the event will no longer function to
>>>> build an editor on top of. At least not without combining it with mutation
>>>> observers and other technologies.
>>>>
>>>> Having a model where IME events are cancelable is really something we
>>>> all hoped to do in a future version of the spec. If the situation is such
>>>> that Chromium/Android would prefer to do this right now, whereas Apple
>>>> prefers to stay with the non-cancelable IME-model then that would not seem
>>>> like terribly different models.
>>>>
>>>>
>>>>>
>>>>>
>>>>>>
>>>>>>>
>>>>>> > 4. deleteCompositionText -- removed (fx "help"): It removed the
>>>>>>> text from the underline and it may not be the entire composition.
>>>>>>>
>>>>>>> > 5. insertFromComposition -- adds "help" permanently, and gives the
>>>>>>> JS the opposrtunity to rebuild the complex HTML structure lost in step 1.
>>>>>>>
>>>>>>> Step 3 is already permanent.  The fact that there is still an active
>>>>>>> composition underline at the time of step 3 does not make it somehow
>>>>>>> temporary.  It has already been injected into the DOM and caused all side
>>>>>>> effects.  So the JS should rebuild its complex structure at time of event
>>>>>>> 3, and events 4/5 are redundant.
>>>>>>>
>>>>>>
>>>>>> When a composition starts, they temporarily pause most activities
>>>>>> because they are not really allowed to intervene. Then, once the
>>>>>> composition is finished, they deal with the finished text.  input. For
>>>>>> example, in step 5 they may reapply some partial styling that was lost in
>>>>>> step 1.
>>>>>>
>>>>>
>>>>> "compositionend" event is fired at the same timing that steps 4/5
>>>>> would be, isn't it sufficient for this use case?
>>>>>
>>>>
>>>> There is one problem in that compositionend with data="" could mean the
>>>> composition has been cancelled or that the final compositionstring is
>>>> empty. The distinction between 4 and 5 is needed if 2, 3 & 4 are
>>>> non-cancelable. If they are cancelable, then one should be able to combine
>>>> 1 and 2, and skip 4 and 5 and the JS would continously make any adjustments
>>>> it needs to the DOM (while taking into consideration what the
>>>> compositionstring is at any moment).
>>>>
>>>>
>>>>>
>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> The reason you state for the proposed change is that "On Android
>>>>>>>> it’s very common for IME to start composition whenever a cursor is moved
>>>>>>>> onto a word.  This is primarily intended to passively show an underline to
>>>>>>>> the user (to indicate what would be replaced *if* user selects an alternate
>>>>>>>> spelling)."
>>>>>>>>
>>>>>>>> If such tapping leads to actual DOM changes, then  the JS needs to
>>>>>>>> know about those(step 1/2). But if you don't change the DOM immediately and
>>>>>>>> only do so at a later stage when you "really" start the composition, then
>>>>>>>> you can also wait until that stage.
>>>>>>>>
>>>>>>>>
>>>>>>>> So these steps are needed on allplattforms, no matter whether or
>>>>>>>> not you change how IMEs work on Android.
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>> Adding aelias@ to make sure he is able to make the call on the
>>>>>>>>> 14th.
>>>>>>>>>
>>>>>>>>> dave.
>>>>>>>>>
>>>>>>>>> On Fri, Feb 3, 2017 at 1:33 AM, Johannes Wilm <
>>>>>>>>> johannes@fiduswriter.org> wrote:
>>>>>>>>>
>>>>>>>>>> Could we get the person or people who came up with these changes
>>>>>>>>>> due to Android to participate in the call on the 14th? I think it's very
>>>>>>>>>> important to have them invovled in this discussion.
>>>>>>>>>>
>>>>>>>>>> On Fri, Feb 3, 2017 at 10:21 AM, Johannes Wilm <
>>>>>>>>>> johannes@fiduswriter.org> wrote:
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Fri, Feb 3, 2017 at 10:18 AM, Johannes Wilm <
>>>>>>>>>>> johannes@fiduswriter.org> wrote:
>>>>>>>>>>> ...
>>>>>>>>>>>
>>>>>>>>>>>> Right now I cannot see any purpose of the beforeinput event
>>>>>>>>>>>> with these changes applied, but maybe I am missing something?
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> I need to restarct that. I guess it could still be useful to
>>>>>>>>>>> stop native bold/italic buttons from making their own, non-controlled DOM
>>>>>>>>>>> changes. It's just for text input that it is irrelevant. And IME will
>>>>>>>>>>> continue to be a mess.
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Johannes Wilm
>>>>>>>>>>> Fidus Writer
>>>>>>>>>>> http://www.fiduswriter.org
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Johannes Wilm
>>>>>>>>>> Fidus Writer
>>>>>>>>>> http://www.fiduswriter.org
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Johannes Wilm
>>>>>>>> Fidus Writer
>>>>>>>> http://www.fiduswriter.org
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Johannes Wilm
>>>>>> Fidus Writer
>>>>>> http://www.fiduswriter.org
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Johannes Wilm
>>>> Fidus Writer
>>>> http://www.fiduswriter.org
>>>>
>>>
>>>
>>
>>
>> --
>> Johannes Wilm
>> Fidus Writer
>> http://www.fiduswriter.org
>>
>
>


-- 
Johannes Wilm
Fidus Writer
http://www.fiduswriter.org