Re: IME events

Hello,
First, I should have made my goal clearer.  As Nakano-san mentioned, it's
true that we are missing some features sometimes on particular platforms and
sometimes all platforms for now.  What I am proposing is to have the
consensus of the interface to address all challenges on all platforms by
getting the support of platform vendors.  So, this is the mid to long-term
goal.

So, what I'm writing here is not the idea to solve something right now,
rather to have the better understanding the common requirements.

On Tue, Jun 30, 2009 at 1:57 PM, Masayuki Nakano <masayuki@d-toybox.com>wrote:

> Sorry for the delay.
>
> On 2009/06/23 9:31, Takuya Oikawa wrote:
>
>> On Sat, Jun 20, 2009 at 1:51 AM, Masayuki Nakano <masayuki@d-toybox.com
>> <mailto:masayuki@d-toybox.com>> wrote:
>>
>>    On 2009/06/18 19:11, Takuya Oikawa wrote:
>>
>>        On Tue, Jun 16, 2009 at 11:56 AM, Masayuki
>>        Nakano<masayuki@d-toybox.com <mailto:masayuki@d-toybox.com>>
>>  wrote:
>>
>>
>>
>>            On 2009/06/15 13:52, Daniel Danilatos wrote:
>>
>>
>>                IME API
>>
>>                * Control the IME Candidate window (get/set position,
>>                ability to prevent
>>                it from showing up at all, etc)
>>
>>
>>            Applications cannot control the candidate window position
>>            via TSF, probably. However, if browsers implement to render
>>            the candidate window themselves, we can. But gecko doesn't
>>            have such plan. Because most users don't happy. The
>>            candidate window should have native looks of the IME.
>>
>>
>>        Yes, it's true.  TSF's UIless mode
>>        (http://msdn.microsoft.com/en-us/library/aa966970.aspx) provides
>> you
>>        to disable the candidate window.  So, the browsers can implement to
>>        render the candidate window themselves.
>>
>>        For most of the cases, as you said, there is not so strong need to
>>        have browsers render the candidate window.  However, if the
>> browsers
>>        also have the function to control the candidate window and its
>>        function is published as JavaScript to Web applications, Web
>>        applications may use it.  For example, as some might have already
>>        pointed out, Search suggest (i.e. Google Suggest or Yahoo! Suggest)
>>        cannot handle the conflict of UI (overwrapping UIs of the suggest
>>        window and IME's candidate window) now, but if Browsers' JavaScript
>>        can control the behavior, they will probably use it.
>>
>>
>>    The candidate window is not just a list of the candidate strings at
>>    ATOK. ATOK gives some advices of the selected word to the users. And
>>    also users can access to the word meaning of dictionaries. So, I
>>    think that the Web App developers should not kill such IME features.
>>
>>
>> I know.  MS-IME also has such features (annotations, usage/meanings,
>> etc.).  However, the most basic feature that all IMEs should have is the
>> Japanese conversion.  I don't say that Web Apps can kill those
>> additional/rich functions.  What I want to say is that it would be
>> better to have Web Apps (via Browsers) control the behavior of IME
>> including the control of the position of the candidate window or even
>> enabling/disabling the candidate window.  If we have such features, even
>> in the worst case that we don't have any other solutions, Web Apps can
>> still give users the Japanese conversion candidates and unconflicting
>> UI.  This is my intention behind this proposal.
>>
>
> I meant, if we want to control the candidate window position under TSF,
> don't we need to draw the the candidate window by ourselves? If so, we may
> not be able to provide the advanced features, right? And also the feature
> chooses the Web App usable OS/environment. Maybe, we cannot specify the
> candidate window position on Cocoa, so, it means that the such Web apps can
> be unusable/unuseful on Mac.
>

Again, what I want to mention here is to give the option to control the
behavior of IME to Web Apps via Browsers.  It's totally up to Web Apps to
decide whether they control IME or not.  Yes, it's true that Web Apps need
to draw the candidate window or blend candidates into their own UIs.  Or,
Web Apps can just disable the candidate window but users are still able to
convert to Japanese (or Chinese/Korean) inline.  The advanced features are
not supported, but conime (Console IME) also limits the capability to just
the basic feature which is conversion, and doesn't provide any additional
features.  So, we can assume users can live with it under specific
condition.  And, it's totally up to Web Apps that should understand the
users' requirements.  If Web Apps think it's required to support advanced
features, they should NOT disable the candidate window.

Regard the platform support, I agreed that the feature should not different
from platform to platform.  So, in order to give the control of IME to
browsers, major platforms should expose the interface to browsers.


>
>         Or, as TSF's UIless mode is mostly used by game titles, if
>> web-based
>>        (browser-based) applications become popular, they also would like
>> to
>>        disable the IME's candidate window.
>>
>>
>>    I'm not sure that it makes the users happy. Do the users input
>>    comfortable without IME's candidate window?
>>
>>
>> For such limited scenario, users seem happy as their main goal is to
>> play game titles where it's very rare to have richer functionalities of
>> IMEs.  What they need to do with IMEs is to type Japanese.
>>
>
> The users wanted to input some CJK text via the IME, right? At that time,
> are the users really happy by the candidate window to be disabled? Isn't the
> disabling the candidate window wanted only by the game designers?


Yes, by the game designers who should understand the requirements of their
users.  There is always a trade off.  In the case you don't have the best
solution, for this case, showing the candidate window and avoiding UI breaks
at the same time, compromising something could be the option.


> I think the candidate window isn't a part of applications. It is a part of
> the system. E.g., IMEs on Cocoa and TSF decide the position of the candidate
> window from the caret position or composition string rect.
>
> I feel that this means IMEs don't want to be touched to their UI from
> applications.


As long as there is any way to avoid the UI conflict between WebApps UI and
IME's candidate window such as "Suggest features" on several web sites, it's
okay not to give the ability to Browser to control the behavior of IME.
 However, I cannot think of any other solution.

And, the fact that there is an interface on Windows (TSF) called "UI less
mode" shows that this is the common requirements not only for Browsers, and
apps should be able to have the ability to control (enable/disable) the
candidate window.

http://msdn.microsoft.com/en-us/library/aa966970(VS.85).aspx


>
>
> NOTE: I think the limitations of each major platforms (the list can include
> some misunderstandings)
>
> * IMM32
>  - ImmSetCandidateWindow can specify the candidate window position.
>  - Can hide the candidate window at WM_IME_SETCONTEXT message (i.e., cannot
> hide it dynamically (?))
>
> * TSF
>  - Cannot control the candidate window position under non-UIless mode.
>  - Cannot hide the candidate window under non-UIless mode.
>  - UIless mode might not be supported by some TIPs.
>  - Can hide the candidate window under UIless mode.
>  - Cannot control the TIP's candidate window position under UIless mode.
> (?)
>  - Can control the candidate window position if a browser renders the
> candidate window itself, but some advanced features may not be able to be
> supported at this time. (?)
>
> * GTK2
>  - gtk_im_context_set_cursor_location can control the candidate window
> position.
>  - Cannot hide the candidate window. (?)
>
> * Cocoa (including Input Method Kit Framework, supported 10.5 and later)
>  - Cannot control the candidate window position.
>  - Cannot hide the candidate window.
>
>
>             And also we cannot control it on Cocoa, probably. Cocoa
>>            applications should implement |firstRectForCharacterRange|
>>            of NSTextInput protocol. And IMEs can decide the candidate
>>            window position by its result.
>>
>>
>>        I'm not sure the implementation of Cocoa.  What I discussed with
>>        Daniel was that we should consider both short-term solution and
>>        long-term solution.  For the latter, we should involve platform/OS
>>        developers who are responsible for input framework and browser
>>        vendors
>>        to discuss the use cases and solutions.  We are aware that it takes
>>        long.
>>
>>
>>    Right.
>>
>>    Basically, I believe that we should not limit/change the behavior of
>>    IMEs more than the *users* needed. There are many IMEs, IME
>>    frameworks, OSs and devices. I think that some IME APIs for web
>>    applications cannot support on some environments that includes new
>>    products that will be coming in the future. Web applications should
>>    not choose the environments by the IME management.
>>
>>
>> Agreed. That's why I think we need to have some consensus or standard
>> for all of these different layers even though it takes long.  I think
>> it's okay to have Web Apps control the behavior of IMEs because local
>> applications can do today.  Of course, having such functionalities in
>> Web Apps (via Browsers) don't mean that it's recommended.  However, for
>> some scenarios which controlling the IMEs' behaviors is the only option,
>> we should have any means, I think.
>>
>
> --
> Masayuki Nakano <masayuki@d-toybox.com>
> Manager, Internationalization, Mozilla Japan.
>