Re: Pointer lock spec

Thank you for the well though out points, Florian.

On Fri, Feb 27, 2015 at 12:22 AM, Florian Bösch <pyalot@gmail.com> wrote:

> I'd like to comment on the pointer lock functionality some.
>
> 12.4 notes that capturing a (a native) pointer inside of a rectangle is
> difficult. I've done some research into this topic and I can attest that
> it's not straightforward. Some platforms have support for this semantics,
> others (I think it was OSX and Windows) would require repositioning the
> pointer in an event loop which has some drawbacks.
>
> 12.5 states that there's difficulties in translation of
> high-resolution/unaccelerated mouse pointer values to sensible units. While
> I appreciate that concern, I don't think it matters nearly as much. The
> first reason is that the user interaction with the pointing device isn't
> aspect-distorted in the first place (that is a horizontal movement in the
> pointing device covers the same distance on screen as a vertical movement).
> What we're really discussing is mouse-sensitivity then, which could vary
> greatly once mouse acceleration and sensitivity (by the OS) is taken out of
> the picture. However, applications can (and will) provide their own
> settings for these values. The reason to bypass acceleration/sensitivity of
> the OS for a pointer is specifically because that treatment often doesn't
> make sense (say for a camera control function in a 3D environment).
>

You raised this point in 2011, resulting in my adding this spec section you
reference. The relevant bit being:
"""
... a concern of specifying what units mouse movement data are provided in.
This specification defines .movementX/Y precisely as the same values that
could be recorded when the mouse is not under lock by changes in
.screenX/Y. Implementations across multiple user agents and operating
systems will easily be able to meet that requirement and provide
application developers and users with a consistent experience. Further,
users are expected to have already configured the full system of hardware
input and operating system options resulting in a comfortable control the
system mouse cursor. By specifying .movementX/Y in the same units mouse
lock API applications will be instantly usable to all users because they
have already settled their preferences.
"""

As an application developer I agree the unprocessed data would be nice to
have, but I don't see it as essential. The benefits of system calibrated
movement are high. Not requiring users to configure every application is
good. And, as the Chrome implementation maintainer who has been in
conversation with several application developers (Unity, Unreal,
PlayCanvas, GooTechnologies, Verold, come to mind easily) this has not been
raised yet as an issue.

The point is still valid, but difficult to engineer a specification
resolution for and my response remains the same as before. Increased
fidelity is appropriate in a follow up specification. I've added a concrete
section for this to:
https://www.w3.org/wiki/Webapps/PointerLockFeatures


> More general comments:
>
> The specification does not discuss latency and pointer emulation. This is
> a significant concern, as routinely native pointers have a latency of
> around 20-30ms, whereas the RTT for pointer-captured events and blit to
> display have latencies between 70-150ms. This makes it difficult to use the
> pointer lock API for emulating a mouse pointer (or indeed to use it for any
> kind of pointing that requires accuracy and speed, like say FPS shooters).
>

I'm not certain how to address this in the specification. I agree that poor
rendering latency will impact the use of an application drawn cursor or
other use, and that some applications and devices may have poor
experiences. That said, what should we change in the specification, or what
notes would you suggest are added to FAQ on this topic?


>
> A particular concern with a frequent usecase (3d interaction by mouse
> drag) is the presence of pointerlock messages. Because we need native
> pointers (in order to interact with DOM elements, to avoid high pointer
> input lag and to allow a user to interact with things outside of the
> viewport (such as tabs, file managers etc.)), it is desirable to stay out
> of a pointerlock mode for general interaction. But when the application
> features a 3D viewport (such as is found in CAD, 3D modelling etc.)  it is
> desirable to enter pointer lock for mouse dragging in those viewports, as
> it avoids text selection and screen border bump issues. A common way to do
> this is to capture the mouse on mouse down, and release capture on mouse
> up. However, this usecase faces some hurdles:
>
>    1. If the user never entered pointerlock on the site, then pointerlock
>    will not be entered until the user clicked "allow" in the confirmation
>    dialog (this interaction is confusing to users)
>
> As with many web platform features that require a user to accept the
permission, the best practice is to educate users why a permission is
necessary. Also, the permission can be established ahead of time.
e.g.
- User visits site, and enters application.
- Anticipating need for pointer lock, the application displays explanatory
text and invites user to click a button for the initial prompt.
requestPointerLock is called.
- User accepts the pointer lock permission.
- Application immediately releases lock, as it won't be needed until later.

Alternatively:
- User visits site, and enters application.
- User attempts a drag operation
- Application attempts to lock pointer, but lock is not entered.
- Application handles mouse move events until mouseup. Doing a best effort
action that the drag was used for (e.g. panning viewport).
- Application shows a message, "Hey, when you drag we'd like to make the
experience better, but you'll need to accept the permission prompt".


>    1. When he clicks allow, the mouse will be captured, but the release
>    event might not have been dispatched, so that the event order suggests the
>    pointer is captured, though that would run counter the semantic (I'm not
>    sure what the current UA behavior there is, but this was the case in some
>    UAs as of a year ago)
>
> That would be an implementation bug, the specification doesn't alter the
dispatching of mouseup. Chrome doesn't seem to have this bug now.

<html>
<body onload="
document.body.onclick = function () {
output.innerHTML += 'click\n';
}
document.body.onmousedown = function () {
output.innerHTML += 'down\n';
document.body.requestPointerLock();
}
document.body.onmouseup = function () {
output.innerHTML += 'up\n';
}
">
<pre id="output">

outputs
  down
  up
  click
and the permission bubble is shown.

>
>    1. Regardless of previous pointer capture confirmation, some UAs
>    choose to display a popup with information everytime somebody enters
>    pointerlock, which makes frequenty entry/exit fr
>
>
This is a UA implementation decision, and it may change over time. E.g.
Chrome may decide that pointer lock while in fullscreen and with keyboard
lock is too risky to not show some reminder that pointer lock has been
re-entered.

Today, Chrome does not show a reminder UI when pointer lock is entered it
if was previously exited by exitPointerLock. This was used by cloudparty.com
(now defunct) as they entered pointer lock only during a drag, and did so
without annoying UI. You can test this today on
http://scheib.github.io/HTMLMisc/PointerLockAndFullscreen.html by entering
a lock, using "Unlock Pointer" button at top to leave, and then
re-entering.

I'd also like to expand a bit on the topic of high resolution pointer
> information, which means both high temporal and high spatial resolution.
> For example The Razer Ouroboros mouse (
> http://www.razerzone.com/gaming-mice/razer-ouroboros). It supports a
> polling frequency of 1000hz and has a resolution of 8200dpi. OSes (after
> they're done mangling pointers trough their machinery) clamp pointer
> positions to pixels, and they clamp frequency of events at least to pixel
> crossover of the pointer and to no more than 60hz or so. It would be highly
> desirable to get sub-pixel accuracy of a pointing device (as is supported
> by nearly all mice produced in the last 10 years) as well as higher spatial
> frequency than the one imposed by the OSes pointer machinations. These
> things are desirable because they allow an application that runs say a 60hz
> display frequency, to temporally sub-sample a pointer location from say 16
> different events. So even though at the start and end of a frame, the
> pointer might be in the same position again, a movement intra-frame could
> still be registered, smoothed and used. More generally this is useful to
> provide a smooth feel and rapid response to applications that deal with a
> need to use pointers for fast and/or accurate pointing (such as for
> instance FPS shooters).
>
> I agree this would be great, we discussed this in 2011 as well. It is
included in https://www.w3.org/wiki/Webapps/PointerLockFeatures for
consideration in a future iteration of the spec. It would be good to note
any other developers who would find that useful, especially if they have a
deployed product.


> On Thu, Feb 26, 2015 at 8:21 PM, Vincent Scheib <scheib@google.com> wrote:
>
>> Thanks, Philip, changes made.
>>
>> On Thu, Feb 26, 2015 at 10:58 AM, Philip Jägenstedt <philipj@opera.com>
>> wrote:
>>
>>> Also, the EventHandler type should not be nullable, it's already
>>> typedef'd to be nullable in https://html.spec.whatwg.org/#eventhandler
>>>
>>> On Fri, Feb 27, 2015 at 1:56 AM, Philip Jägenstedt <philipj@opera.com>
>>> wrote:
>>> > https://dvcs.w3.org/hg/pointerlock/raw-file/default/index.html
>>> >
>>> > The section "Element Interface" should be "Extensions to the Element
>>> > Interface" to match the other sections, and because it's true :)
>>> >
>>> > Philip
>>>
>>
>>
>