Re: 2 Proposals for Minimum Viable InputEvent

One variant of this that we could consider instead of making all
insert*/delete* uncancellable would be just making the text
insertion/deletion ones uncancellable.

I'm not sure exactly what that list should be, but I think it'd be
something like: "insertText", "insertReplacementText",
"insertCompositionText", "insertFromComposition", "deleteCompositionText",
"deleteByComposition".

I'm unsure if we also need to include: "deleteWordForward",
"deleteWordBackward", "deleteSoftLineForward", "deleteSoftLineBackward",
"deleteEntireSoftLine", "deleteHardLineForward", "deleteHardLineBackward".

On Tue, Feb 7, 2017 at 8:18 PM Ojan Vafai <ojan@google.com> wrote:

> Oh, and #5, our intention is not to violate the spec, but rather to get
> the spec changed so that all browsers can agree on behavior here. Even in
> the case that Alexandre mentioned where iOS will deliver two events and
> Android one, the event types would be the same (namely
> ‘insertCompositionText’). So the interoperability impact will be fairly
> minimal.
>
> The big difference here is that for IMEs that don't change the markup at
> start of composition, they're not forced to first delete the text and
> reinsert it because of having a delete followed by an insert for what is
> essentially just an insert.
>
> On Tue, Feb 7, 2017 at 8:12 PM Ojan Vafai <ojan@google.com> wrote:
>
> This discussion has gotten rather complicated, but I want to call out a
> few points.
>
>    1. In all these proposals, JS always knows about changes to the DOM
>    before they happen via the beforeInput event and some of those cases are
>    cancellable. In the latest proposal that started this thread, we're
>    proposing that more of those cases are not cancellable. We think this will
>    lead to a simpler system for web authors to understand (all insert*/delete*
>    are not cancellable) and something that we can implement more efficiently.
>    2. One of the use cases for beforeInput is JS libraries that want to
>    do the DOM modifications themselves. That's certainly not the only use case
>    though. Right now on Android, there's no way to know which text is going to
>    be replaced by which text when the user does an editing operation. All
>    proposals here address that problem.
>    3. If we can find a way to make the insert*/delete* types cancellable,
>    it will most likely be backwards-compatible to make them cancellable in the
>    future. In the meantime, we can get a v1 of beforeInput out there now and
>    solve the basic use case in #2.
>    4. The only way to make interaction with IMEs truly sane is to expose
>    a first-class IME API to talk to IMEs. We've heard from JS library authors
>    in this group that they aren't interested in adding that complexity to
>    their libraries anytime soon. So, we're not prioritizing designing those
>    API. The sad reality is that beforeInput will have to have some warts due
>    to IME interaction on *desktop* platforms not allowing cancellation. We
>    believe the new proposal reduces warts by simplifying what is or isn't
>    cancellable in a way where a naive developer is more likely to support
>    non-latin languages by accident. This is obviously a subjective intuition,
>    so I doubt we'll get anywhere by arguing about whether this is a valid
>    belief.
>
>
> On Mon, Feb 6, 2017 at 7:45 PM Johannes Wilm <johannes@fiduswriter.org>
> wrote:
>
>
>
> On Tue, Feb 7, 2017 at 4:00 AM, Alexandre Elias <aelias@google.com> wrote:
>
> On Mon, Feb 6, 2017 at 6:18 PM, Johannes Wilm <johannes@fiduswriter.org>
> wrote:
>
> On Tue, Feb 7, 2017 at 12:23 AM, Alexandre Elias <aelias@google.com>
> wrote:
>
> Hi, I'm the maintainer of IME on Android.  I'll be happy to attend the
> conference call next week.  As you've surmised, Android does need to
> perform a DOM change to merge two nodes "later".  However, I disagree that
> those 5 steps are somehow needed.  Our plan is to send a single
> insertCompositionText beforeinput event instead of 5 events.
>
>
> Nice to meet you Alexandre!
>
> It's a bit unfrotunate that you haven't been able to participate in the
> last 3 years of discussions about this subject (I think some people have
> even been discussing this for more than 8 years), and it seems that the
> mailing list isn't as alive any more as it once used to be, but I'll do my
> best to give the reasoning why we came to the conclusion that these events
> are needed. I would also recommend consulting the various open and closed
> issues on github and to consult the archive of the mailing list. Also,
> various Google people have been behind a lot if not most of the various
> various concepts, so they can probably give you more background about this
> as well.
>
>
>
> Arguments why each is not needed:
>
>  > 1. deleteByComposition --This step is only needed when recomposing. A
> composed word can go across element boundaries, so this may delete more
> than just contents of a text node. (fx " He<b>ll</b><i>o fish</i>!"). This
> step needs to be executed at the moment your Android IME changes the DOM,
> whether this is at the moment the user taps the word and the underline
> appears or at a later stage.
>
> There's no need for a separate "delete" event, because
> insertion-to-replace-range is general enough to represent a deletion range
> as well.  I prefer to reserve "delete" events exclusively for intended IME
> deletions (e.g. select word and press backspace).
>
>
> But I am a little confused: From the proposal Chong sent in, it sounded
> like you just wouldn't have any beforeinput event for this initial
> deletion. That would create a range of different problems, such as:
>
>
> I agree we should not hide any editing information from JS.  I am saying
> that event 2/3 can encompass the deletion.  An implied, wrong assumption
> underlying your argument seems to be that an insertion operation can only
> ever add new text but not remove existing text.
>
>
>
> No, that must be a misunderstanding. I don't know why you would think
> that. As I said previsouly, if  beforeinput events during IME are
> cancelable, 1 (deletion) and 2 (addition cshould be combinable into one
> single event,
>
>
> I don't quite get what you mean by combining steps two and three. Step two
> was the reinsertion of the initial composition string into the DOM. Step
> three is the change during the composition. There will usually be many step
> threes, but only one step two.
>
> The reason there is a step one and two at all is that the IME converts the
> composition string from being something that can contain a complex
> html-structure to just be  a plaintext string (in most cases anyway).
>
> So step 1 will remove "He<b>l</b>p" and step 2 will re-add it as "Help".
> Combining step 1 + 2 should work if step 2 is also cancelable. Combining
> step 2 and 3, or maybe 1 and 2 and 3 (?) would mean that we go straight
> from "He<b>l</b>p" to "Hel". and then the next two step 3 would add "l" and
> "o". In many cases the JS will probably be able to guess what the
> individual parts of this combination of 1/2/3 initially were (what part was
> just about turning it into a plaintext and which part was then about
> modifying the plaintext), but it would seem a bit meshed and I'm not sure
> one always can determine it. Nevertheless, if iME input is cancelable, then
> this does sound better than what was on the table previosuly.
>
>
>
>
>   In reality, insertions are always *replacements* of the content of the
> caret/selection/composition with text (for example, if you select text with
> the mouse and then type "a" on a physical keyboard, no deletion event is
> sent).
>
>
> right. That'sw hy we have targetranges. But the difference for keyboard
> inout is that it is cancelable and there is no automatic normalization to
> plaintext. This is different than the situation with IME.
>
>
>
> * The more advanced editors today are moving toward a model where they
> hold a json version of the document which is separate from the DOM. They
> only make adjustments to the DOM when needed, and if the user changes
> something about the DOM, they make adjustments to the  json document. The
> json version is the authoritative copy of the document. For this to work,
> they need to know about any change to the DOM. If you make user-initiated
> DOM changes that are not causing any beforeinput event at all, the
> beforeinput event is somewhat useless, aand they will have to continue to
> monitor user initiated DOM-changes in other ways.
>
> * In a collaborative editor, if you make a change to the DOM of one user,
> and the JS is not being made aware of this change, then it won't notify the
> collaborators of this change, and so the collaborator will ahve different
> versions of the document. Again, the point of beforeinput events will be
> gone. JS develoeprs will use mutation observers or some other technology
> instead.
>
>
>
>
>
> > 2. insertCompositionText -- Also this step only needs to happen at the
> moment only for recompositions. It reinserts the word from step one, but
> this time as plaintext in one element (fx "Hello" => "<i>Hello fish</i>")
>
> On Android because of the "late" timing, this merge operation would happen
> in the same message loop task as step 3, so there is no need for this
> event.  We prefer to go directly from "He<b>ll</b><i>o fish</i>" ->
> "<i>Help<i>".
>
>
>
> Yes, in a perfect world, where all IME-related events are cancelable, this
> could be done and would be preferable, as for example Yosin has pointed out
> ( https://github.com/w3c/input-events/issues/45#issuecomment-261427648 ).
> We had a long discussion during the fall of 2015 about this, and among
> other ones several developers from Chromium eventually convinced us here
> that IME-events cannot be guaranteed to be cancelable. Even moving the
> caret to somewhere else during the composition may somehow crash the IME
> (hwo this is possible is a bit of a mystery to me as a JS developer).
>
> One could then say ok, combine 1 and 2 and make them both non-cancelable.
> The problem with that is that the browser doesn't always know the seamntic
> meaning of the various DOM elements used within a document and it may end
> up making decisions of what to delete that are problematic.
>
>
> I've been trying to figure out where these claims about IME from some
> Chromium team members came from exactly and haven't figured it out, nobody
> has a precise memory.  They seem to be based on a vague fears rather than
> concrete knowledge of how IMEs behave.  I agree with your instinct that the
> assumptions behind it don't make too much sense, that's why I think we
> should hold off before we commit to a complicated model based on those
> assumptions.
>
>
> Well, maybe you guys need to discuss that internally. Until now what we
> said here is that we will assume the worst case scenario (that IMEs not
> being allowed to directly modify the DOM will lead to them crashing) and
> then in a future version to hopefully get one that doesn't crash.
>
> Btw -- just to make it clear because this seems to have been a common
> misconception between JS developers and browser developers in the past:
> When we ask for these things to be cancelable, in 80% of cases this is not
> because we want to stop the user from doing things. It's more than we want
> to do the corresponding DOM changes in JS code, because they are specific
> to the JS editor the user is currently using and they may not necessarily
> make sense to the browser.
>
>
>
>
>
>
>
>
> However, rniwa@apple.com has told us that on OS X, the standard platform
> behavior is to merge the nodes at time of composition start.  Therefore, on
> OS X step 2 is still needed and there would be 2 beforeinput events in
> total, but on Android there would be only 1.  This represents a substantive
> platform difference that I think it makes sense to allow to differ in the
> spec.  I don't think we should try to artificially synchronize the
> platforms by always sending 2 events -- on Android this would simply result
> in unnecessary work and side effects that would never be seen by the user.
>
>
> I think you should instead consider making this two events on Android.
> Android is really great, but text editing in Chrome on Android is really
> problematic especially when it comes to IME recomposition, as I am sure you
> are aware. I have tried various keyboard, and but even on mainstream sites
> such as Facebook, using a Google-provided keyboard, recomposition tends to
> just mess up entire sentences, throw random words around, deleting or
> merging words that are close to what one is trying to recompose, etc.
>
>
> Right, we are in process of fixing all of these bugs and expect to get the
> remaining ones fixed in M58/M59 timeframe.  Anyway, this doesn't address my
> point.  Personally, I believe that firing additional unnecessary events
> will prove harmful to both UAs and JS editors -- in reality an atomic IME
> change that just happened, but you are proposing to drip-feed it into many
> separate events.  Your proposal doesn't provide JS any additional
> information (it just splits information into pieces), so therefore I don't
> think it can possibly be needed for any use case either.
>
>
>
>
> Speed is not as much a concern anyway because it all depends on the human
> who is interacting with it, but in addition these really major issues mean
> that recomposition as it exists on Android as of today is not very useful.
> In addition there are the various issues that JS developers have complained
> about for a while: no compositionstart event, etc. . I am sure you know the
> list of grievances with it all too well.
>
>
> We should always be firing compositionStart today.  Feel free to file a
> bug against me if you know of a case where we don't.
>
>
> I recently sent a list of prioritized bugs from several JS editor projects
> to the Chromium team that they had requested from em. I assume they
> forwarded those issues relevant to Android to you.
>
>
>
>
> Not knowing the details, but I could imagine you can check for event
> listeners, and only if those are present you make it two events.
>
> But as an alternative, and to try to get a bit closer to both your and
> Yosin's position, how about this:  We make two IME models, one for IMEs
> where the beforeinput events can be cancelled in which step 1 and 2 are
> merged and one where they cannot be cancelled where they are separate.
>
>
> Disagree, we should keep things as simple as possible.  I want us to fire
> a minimum of events and avoid complicated logic forking.  I don't want us
> to introduce permanent complexity into the platform for the sake of a
> short-term compromise.
>
>
> Well, you said that Apple does one thing and you do another. So either the
> two of you can agree or you don't. If you cannot agree, we'll have to have
> two different models.
>
> In theis scenario, if the Chrome model consists of just cutting out 4 of 5
> events on Android, then as I tried to explain in the last few emails and
> the comments on the proposal doc, the event will no longer function to
> build an editor on top of. At least not without combining it with mutation
> observers and other technologies.
>
> Having a model where IME events are cancelable is really something we all
> hoped to do in a future version of the spec. If the situation is such that
> Chromium/Android would prefer to do this right now, whereas Apple prefers
> to stay with the non-cancelable IME-model then that would not seem like
> terribly different models.
>
>
>
>
>
>
> > 4. deleteCompositionText -- removed (fx "help"): It removed the text
> from the underline and it may not be the entire composition.
>
> > 5. insertFromComposition -- adds "help" permanently, and gives the JS
> the opposrtunity to rebuild the complex HTML structure lost in step 1.
>
> Step 3 is already permanent.  The fact that there is still an active
> composition underline at the time of step 3 does not make it somehow
> temporary.  It has already been injected into the DOM and caused all side
> effects.  So the JS should rebuild its complex structure at time of event
> 3, and events 4/5 are redundant.
>
>
> When a composition starts, they temporarily pause most activities because
> they are not really allowed to intervene. Then, once the composition is
> finished, they deal with the finished text.  input. For example, in step 5
> they may reapply some partial styling that was lost in step 1.
>
>
> "compositionend" event is fired at the same timing that steps 4/5 would
> be, isn't it sufficient for this use case?
>
>
> There is one problem in that compositionend with data="" could mean the
> composition has been cancelled or that the final compositionstring is
> empty. The distinction between 4 and 5 is needed if 2, 3 & 4 are
> non-cancelable. If they are cancelable, then one should be able to combine
> 1 and 2, and skip 4 and 5 and the JS would continously make any adjustments
> it needs to the DOM (while taking into consideration what the
> compositionstring is at any moment).
>
>
>
>
>
>
>
>
>
> The reason you state for the proposed change is that "On Android it’s
> very common for IME to start composition whenever a cursor is moved onto a
> word.  This is primarily intended to passively show an underline to the
> user (to indicate what would be replaced *if* user selects an alternate
> spelling)."
>
> If such tapping leads to actual DOM changes, then  the JS needs to know
> about those(step 1/2). But if you don't change the DOM immediately and only
> do so at a later stage when you "really" start the composition, then you
> can also wait until that stage.
>
>
> So these steps are needed on allplattforms, no matter whether or not you
> change how IMEs work on Android.
>
>
>
> Adding aelias@ to make sure he is able to make the call on the 14th.
>
> dave.
>
> On Fri, Feb 3, 2017 at 1:33 AM, Johannes Wilm <johannes@fiduswriter.org>
> wrote:
>
> Could we get the person or people who came up with these changes due to
> Android to participate in the call on the 14th? I think it's very important
> to have them invovled in this discussion.
>
> On Fri, Feb 3, 2017 at 10:21 AM, Johannes Wilm <johannes@fiduswriter.org>
> wrote:
>
>
> On Fri, Feb 3, 2017 at 10:18 AM, Johannes Wilm <johannes@fiduswriter.org>
> wrote:
> ...
>
> Right now I cannot see any purpose of the beforeinput event with these
> changes applied, but maybe I am missing something?
>
>
> I need to restarct that. I guess it could still be useful to stop native
> bold/italic buttons from making their own, non-controlled DOM changes. It's
> just for text input that it is irrelevant. And IME will continue to be a
> mess.
>
> --
> Johannes Wilm
> Fidus Writer
> http://www.fiduswriter.org
>
>
>
>
> --
> Johannes Wilm
> Fidus Writer
> http://www.fiduswriter.org
>
>
>
>
>
> --
> Johannes Wilm
> Fidus Writer
> http://www.fiduswriter.org
>
>
>
>
>
> --
> Johannes Wilm
> Fidus Writer
> http://www.fiduswriter.org
>
>
>
>
>
> --
> Johannes Wilm
> Fidus Writer
> http://www.fiduswriter.org
>
>