Re: Mouse Lock

I've updated the draft spec http://goo.gl/9G8pd with these points.

- Removed Touch related components to simplify initial version of
specification and because they do not add the same level of new application
capability as mouse related events.
- MouseLockable Interface moved from element to document, mouselock takes
target as parameter, mouseLocked returns current target element.
- movementX/Y behavior when system cursor reenters window specified as zeros
- lockMouse must be called from the same origin as the top level document
- Cases of target element not being in DOM tree specified to fail or exit
lock
- Mouse user gestures suppressed while under mouse lock
- Client and screen coordinates when under lock report the last value they
had when lock was entered. Clarification on events that should and should
not fire when under mouse lock.
- Added example "Example handling failure to lock"
- Use case rewrite: "View-port panning by moving a mouse cursor against the
bounds of a view-port."
- Merged use cases "Free three dimensional model rotation" & "Move layers of
information without bound limits"
- Added FAQ "Why bundle all functionality (hiding cursor, providing mouse
deltas) instead of using CSS to hide the cursor, always providing delta
values, and offering an API to restrict the cursor movement to a portion of
the webpage?"
- Added FAQ "Why modify MouseEvent and reuse existing mouse events instead
of creating a mousedelta event?"
- Added FAQ "Why separate targets for mouse events under mouse lock and
keyboard input focus?"


On Fri, Aug 26, 2011 at 4:56 PM, Vincent Scheib <scheib@google.com> wrote:

>
> -- What if target is removed from DOM, or is not currently in DOM? --
> Additional issues from Olli Pettay on w3 bug.
> I believe mouse lock should fail to acquire if the target is not in the DOM
> tree, and should be exited if the target of an existing lock is removed from
> the tree.
>
> -- Are user gestures generated while under mouse lock? --
> From a security point of view, should we sanitize the events generated when
> in mouse lock (click, down, up, ...) to be 'synthetic' and not 'user
> gestures'? E.g. in Chrome I believe you can only pop up new windows with
> window.open from script code triggered by a user gesture event. If we don't,
> and assuming security model allowing only mouse lock for DOM within top
> level document origin, then a page could direct user gesture events to any
> DOM element in their origin they wanted. That may be used for a form of
> 'click jacking'.
>
> On Thu, Aug 25, 2011 at 5:54 PM, Vincent Scheib <scheib@google.com> wrote:
>
>> Continuing the discussion of some of the open issues I'm aware of:
>>
>> -- If MouseEvent types are returned under mouse lock, what should
>> .clientX/Y and screenX/Y be? --
>> Spec as drafted states they should be the center of the target element.
>> That's likely a poor idea, as discussed, and a better solution is to
>> freeze .clientX/Y and screenX/Y to whatever their last value from the user
>> agent was.
>>
>> (Pending feedback, I will update spec to state that .clientX/Y .screenX/Y
>> will report the same constant values of the last known cursor position just
>> as mouse lock was entered. This will also be the location the system cursor
>> will be placed at when mouse lock is exited)
>>
>> -- Modify MouseEvent, or a new type returned by new mouse events? --
>> Spec as drafted now has simply added .movementX/Y data members to the
>> MouseEvent type which is returned by mouse events: click, mousedown,
>> mouseup, mouseover, mousemove, mouseout. No new mouse event types are
>> introduced.
>> Several, including Glenn Maynard, Robert O'Callahan have brought up points
>> against this, suggesting different events and types.
>>
>> Let me enumerate options and my perceived pro/cons:
>>
>> Option A: as currently drafted: reuse existing mouse events, make minimal
>> modification to MouseEvent to include movementX/Y valid regardles of mouse
>> lock state, mouseover & mouseout will never be fired under mouse lock.
>>
>> pro 1. minimal change to existing events and types
>> pro 2. movementX/Y available even when mouse is not locked. Purely a
>> convenience, apps could compute similar data by tracking the last position
>> of a mouse event and computing deltas (resetting last position on a
>> mouseover event).
>> pro 3. event handling code that can operate both with and without mouse
>> lock being acquired can do so easily using the same functions and taking the
>> same MouseEvent data type input. e.g. rotateView(mouse_event) could be
>> called from a mousemove event regardless of mouse lock state and still use
>> .movementX/Y.
>>
>> con 1. When under mouse lock, clientX/Y screenX/Y will not offer any
>> useful data - or worse implementations may not follow spec and provide
>> misleading data of the actual hidden cursor location.
>> con 2. Not all mouse events using MouseEvent are appropriate, mouseover
>> and mouseout should be suppressed. Glen continued that it is """probably
>> cleaner to stop firing *all* mouse movement events entirely, as if the mouse
>> isn't moving, and to use a separate "mousedelta" event when locked which
>> only has "deltaX" and "deltaY".""" Robert reiterated that point.
>>
>> Option B: make no use of or modification to any existing mouse event or
>> the MouseEvent type. New mouse events created with unique names for click,
>> down, up, move, e.g. mouselockclick, mouselockdown, mouselockup,
>> mouselockmove. New events return a MouseLockEvent type derived from UIEvent
>> that contains movementX/Y, ctrlKey, shiftKey, altKey, metaKey, button. The
>> events and data types are completely mutually exclusive based on the state
>> of mouse lock.
>>
>> pro 1. (dual of Option A con 1): clientX/Y screenX/Y do not exist under
>> mouse lock.
>> pro 2. (dual of Option A pro 3): Functions designed specifically to
>> operate only under mouse lock, or only when not under mouse lock, are only
>> triggered when appropriate.
>>
>> con 1. (dual of Option A pro 1): Larger change, introducing 4 new events
>> and a new return type.
>> con 2. (dual of Option A pro 2): movementX/Y must be calculated manually
>> by apps when not under lock.
>>
>> Some hybrid could be considered, e.g. only make a new mouselockmove event
>> but reuse the click/down/up events. I don't think that hybrid is worth
>> discussing.
>>
>> My opinion:
>> - Option A con 1 (.client .screen not useful, could be wrong): I don't
>> perceive much damage here, and if implementation are making mistakes much
>> more could go wrong elsewhere, but they don't get new data even with faulty
>> client and screen data.
>> - Option A con 2 (not all events returning MouseEvent are appropriate in
>> mouse lock) isn't substantial... 4 of the events are and they happen to
>> share a MouseEvent type.
>> - I see no logical argument leading us to one or the other option. I'd
>> prefer the minimal change, the convenience of code that can use the same
>> MouseEvent structure, and .movementX/Y even when not in mouse lock.
>>
>> (Pending feedback, I will leave spec at Option A but add clarification
>> about e.g. mouseover/mouseout not being called)
>>
>> -- Separate targets for keyboard and mouse locked events? --
>> Robert: "... is there a need for mouse-lock motion events to go to one
>> element while keyboard input goes elsewhere? ... I'm asking whether we can
>> reuse the existing notion of the "currently focused element" and send mouse
>> events to that element while the mouse is locked."
>>
>> I argue yes, there's a good cause for mouse events to a different target
>> than keyboard input. Status quo enables typing into an input field while
>> mousemove events are sent to different targets the mouse happens to be
>> moving over. In mouse lock, consider a game with a 3D viewport controlled by
>> mouse and a chat window with the active keyboard focus.
>>
>> (Pending feedback, I will leave spec as is)
>>
>> -- .lockmouse() on DOM elements or document? --
>> Robert: "... provide mouse-locking on a per-element basis? It seems to me
>> it would be enough for mouse-locking to be per-DOM-window
>> (or per-DOM-document) and deliver events to the focused element."
>> Jonas Sicking: "But if the mouseLock API was only available on the
>> Document object, then everyone would call it there and there would be no
>> interference between aggregated content. As soon as the lock was granted,
>> anyone that had requested it would be notified."
>>
>> Spec as drafted places .lockMouse() .unlockMouse() and
>> .mouseLocked() interface on DOM elements and allows them to become the
>> target of all mouse events when under mouse lock.
>>
>> It makes sense to move the.lockMouse() .unlockMouse() and .mouseLocked()
>> methods to the document. For me, the main reason is that the query
>> .mouseLocked() should be easy for any code to call to understand the state
>> of user interaction. No special ID or DOM element should be required to make
>> that query.
>>
>> Also, it makes sense to have .mouseLocked() return the current target
>> element of the mouse lock. That enables queries such as "Is the mouse
>> locked? And, if so, do I have the lock or some other widget?".
>>
>> (Pending feedback, I will migrate the MouseLockable Interface from element
>> to the document, update lockMouse to take a target element, and will adjust
>> mouseLocked to return the current target.)
>>
>> -- Automatically release mouse lock on mouseup --
>> Robert proposed an option to act similarly to the .setCapture API and
>> permit a mouse lock based on a user gesture button press that automatically
>> ends when they release the button. This would enable some use cases without
>> requiring permission prompts to users or larger risks.
>>
>> I agree with this goal. I hesitate to place it into v1 of this spec
>> because it helps the minor use cases in windowed mode but we still do not
>> have an implementation solving the major ones. And, to implement this one
>> should also have an implementation of .setCapture, which e.g. Chrome does
>> not now and it is not planned. It is not clear if this feature should live
>> on .lockMouse or on .setCapture. If both were implemented, either API could
>> be augmented fairly easily to offer the hybrid functionality.
>>
>> (Pending feedback, no change to spec)
>>
>> -- movementX/Y should not 'pop' when not under mouse lock and mouse enters
>> window --
>> Upon entering the user agent window what should movementX/Y be? I argue
>> they should be zeroed. Alternatives are to compute delta of mouse cursor's
>> position that was out of window (may be difficult, tiny risk of security
>> issue), or to provide a big jump by computing the delta from the last
>> location the mouse was when it was on top of the window.
>>
>> (Pending feedback, I will add this detail to spec)
>>
>> -- Provide mouse deltas without hiding the cursor, alternatively provide a
>> bounding rectangle for mouse --
>> (See Klaas's recent suggestions)
>> It may be possible, but significant implementation hurdles remain for
>> this. First, we must be able to specify the units that mouse deltas will be
>> provided in. That does not appear straight forward. E.g. the windows API
>> offers only the movements, sampling frequency, and HID id. Offering data in
>> "post ballistics" / accelerated and configured mouse pointer units is
>> straight forward to specify and implement across platforms and
>> implementations of this API. If we find implementation routes we can add
>> access to this data in a later revision, the two API versions are not
>> mutually exclusive and implementation work would be straight forward to
>> share. All use cases are possible even with the 'bundled' proposal as
>> currently spec-ed though they may need to create their own cursor for a
>> visible cursor.
>>
>> -- Multiple locks, or migrating a lock, or iframes --
>> In security discussion with Chrome staff we suggest that .mouseLock is
>> only possible from code executing in the top level document origin. Nested
>> iframes would work only if matching the origin of the top most parent
>> document. If mouse lock is currently held, we see no reason to not allow it
>> to be transferred between elements or plugins of that origin. It is the
>> application developer's responsibility to have cooperating code that avoids
>> cycles of stealing input. The user experience would be oddly multiplexed
>> input across components, but no security threat exists, or even nuisance as
>> the escape logic remains the same. (repeated re-locks prevented as suggested
>> similar to pop-up blockers). Preventing other elements from unlocking and
>> performing their own lock would add complexity to the spec and
>> implementations and seems unneccessary.
>>
>> (Pending feedback, I will add to the spec the restriction of matching the
>> top level document's origin. Spec as is states lock can be transfered at the
>> option of the user agent -- I will change to explicitly allow.)
>>
>>
>