Mouse Lock

I'm moving a conversation thread about the mouse lock specification
from a webkit bug https://bugs.webkit.org/show_bug.cgi?id=68468#c10 to
here:

Alexey Proskuryakov has several questions:

> One thing that stands out to me is MouseLockable interface. Wouldn't we be better off with navigator.lockMouse(), which could potentially be extended to navigator.lockMouse(in Element target) later?

I'm not sure which question this is, perhaps:
- Why have a target element in lockMouse()?
I think the FAQ item "Why separate targets for mouse events under
mouse lock and keyboard input focus?" answers this with "Consider a
game with a 3D view controlled by moving the mouse cursor, while the
user may still chat with other users via a text console. It is
reasonable for the application to accept text input to an element that
is different than where mouse events are being dispatched. This is
similar to preexisting behavior of receiving mousemove events over any
element while typing into a form on a page."

Or, simply, it enables cleaner and more modular application
development. Why not route mouse messages to a target element?

- Why document.lockMouse() instead of navigator.lockMouse()?

Is there a reason navigator would be more appropriate?

> I see that providing an element helps dispatch events. Yet the spec is not consistent anyway - some notifications use DOM events, while others use callback functions.

Mouse events continue to dispatch as they do when not under mouse
lock, it doesn't make sense to change that. If mouse lock status is
lost, an event seems reasonable. Callbacks for the asynchronous
success or failure of a call to lockMouse serve the need without
adding unnecessary new events, and is a pattern already established in
recent specs such as geolocation. Are there any issues with this? You
haven't identified any inconsistencies.

> On an unrelated theme, the definition of movementX/movementY seems too platform dependent to me. The spec says "movementX/Y must continue to provide the change in position of the mouse as when the mouse is unlocked". That's likely not what will work for games given dynamic acceleration on some platforms. Such acceleration would yield unacceptable results if applied to FPS shooter movement control.

On the contrary, this decision is based on offering a consistent
experience on all platforms. The movement is 100% backwards compatible
with mouse events available today. Users on systems calibrate, if
needed, their input devices to a reasonable and comfortable setting.
As they move between platforms and input devices they maintain their
ability to control the mouse cursor in the dimension of screen pixels.
Under moues lock the same data is provided.

On the topic of acceleration and calibration, there seems to only be
speculation that this is harmful. Developers such as Unity and myself
have voiced first hand experience that this is not a concern. Users
who desire no acceleration can configure it the same way they do with
games today (system configuration, drivers, or in game). There is a
substantial FAQ item which covers this topic as well. If you'd like
more details, please see the public-webapps thread.

FAQ item:
"""
Why bundle all functionality (hiding cursor, providing mouse deltas)
instead of using CSS to hide the cursor, always providing delta
values, and offering an API to restrict the cursor movement to a
portion of the webpage?

There are good motivations to provide a more fine grained approach.
E.g. the use case "View-port panning by moving a mouse cursor against
the bounds of a view-port" doesn't require hiding the mouse cursor,
only bounding it and always having delta values available. Also, this
specification defines the movement deltas to be taken from how the
system mouse cursor moves, which incorporates operating system
filtering and acceleration of the mouse movement data. Applications
may desire access to a more raw form of movement data prior to
adjustments appropriate for a mouse cursor. Also, raw data may provide
better than pixel level accuracy for movement, as well as higher
frequency updates. Providing the raw delta movement would also not
require special permission or mode from a user, and for some set of
applications that do not require bounding the cursor may reduce the
security barriers and prompts needed.

There are two justifications for postponing this finer grained
approach. The first is a concern of specifying what units mouse
movement data are provided in. This specification defines .movementX/Y
precisely as the same values that could be recorded when the mouse is
not under lock by changes in .screenX/Y. Implementations across
multiple user-agents and operating systems will easily be able to meet
that requirement and provide application developers and users with a
consistent experience. Further, users are expected to have already
configured the full system of hardware input and operating system
options resulting in a comfortable control the system mouse cursor. By
specifying .movementX/Y in the same units mouse lock API applications
will be instantly usable to all users because they have already
settled their preferences.

Secondly, the implementation of providing movement data and bounding
the mouse cursor is more difficult in the fine grained approach.
Bundling the features together gives implementations freedom to use a
variety of techniques as appropriate on each operating system and is
more practical to implement. Direct APIs do not exist on all platforms
(Win, Mac, Linux) to bound the cursor to a specific rectangle, and
prototypes have not yet been developed to demonstrate building that
behavior by e.g. invisible windows with xlib or manual cursor movement
on Mac. Unaccelerated Delta values have been proposed to be accessed
by reading raw Human Interface Device (HID) data. E.g. WM_INPUT
messages on windows, and USB device apis on Mac / Linux. The challenge
here is interpreting and normalizing the units to some consistent and
specifiable scale. Also, most APIs considered to date are limited to
USB devices.

It would be reasonable to consider adding these capabilities in the
future, as the currently specified mouse lock API would be easy to
continue to support if the finer grained delta and confinement
features were implemented.

The bundled API is selected for implementation practicality, because
the desired use cases are supported, and because it will not conflict
with future improvements as discussed here."""

> I'm surprised by "jog movement over spinner controls" use case. Doesn't that work already? E.g. I can drag the mouse outside of browser window, while <input type=range> will continue to respond. Getting use cases right seems fairly important.

You can not today drag a job control in a single direction with
unlimited input. You will encounter a screen edge. Mouse lock enables
this by locking the mouse for the duration of the drag, e.g. the Maya
DCC tool does this I believe.

> Solution for "Synthetic cursor interaction with HTML DOM UI" use case appears to directly contradict HTML5. Synthetic mouse events should not behave like normal ones, it's a bug in WebKit when they do.

Could you be specific? You do not believe a click event should be able
to be dispatched from JS to another element? What from this section do
you disagree with?

> I see that the spec used to say something about touch events, and no longer does. Regardless of what it used to say, it seems critically important to explain what happens with mouse events that are synthesized from touch events on mobile devices. Should touch capable devices such as phones and tablets just avoid supporting MouseLock?

I agree that the spec should explicitly state the result, since touch
events are quite important. I believe the mouse events generated from
touch gestures under mouse lock should behave the same way mouse
events from a mouse or trackball would. There are issues around mouse
lock mode escape the user agent would need to solve to support mouse
lock if no keyboard is present, but the spec leaves that to the user
agent and future spec versions.

Received on Thursday, 20 October 2011 05:16:05 UTC