RE: 3D Pointers

On Mon, Mar 18, 2013 at 4:09 AM, Arthur Barstow <art.barstow@nokia.com> wrote:
>
> Hi Bill,
>
> FYI, Rick Byers started a related thread "pointerType extensibility" last February <http://lists.w3.org/Archives/Public/public-pointer-events/2013JanMar/0096.html>. During our March 12 call we discussed the extensibility issue again and agreed to:  keep the existing functionality for v1 <http://www.w3.org/2013/03/12-pointerevents-minutes.html#item03>; add the extensibility issue to the v2 feature list <http://www.w3.org/wiki/PointerEvents/UseCasesAndRequirements>.
>
> -ArtB
>
>
> On 3/15/13 7:22 PM, ext Bill Fisher wrote:
>>
>> Hi,
>>
>> I'm wondering if there are any plans to incorporate 3D pointers coming from a motion capture device such as Leap Motion. (video: http://www.youtube.com/watch?v=_d6KuiuteIA , home: https://leapmotion.com)
>>
>> I do not see any such capabilities in the current proposed specification, and this would appear to be a great limitation of the proposed pointer events as we move toward post-touch interfaces based on similar controllers.
>>
>> Leap Motion, for example, has their own API to represent "pointables" in 3D space, and this is made available to web pages through Web Sockets and Node.js via the controller software.
>>
>> But it seems to me like it would be a better idea to create a specification where this information could be exposed to the browser directly.
>>
>> For more info on how Leap Motion is representing 3D pointables in JavaScript, please see:
>> https://github.com/leapmotion/leapjs
>> http://leapmotion.github.com/leapjs/index.html
>> And perhaps most notably: http://leapmotion.github.com/leapjs/Pointable.html

As Art mentioned above,  we've had the related discussion about adding new device types. The model is extensible by new device types and we'll be surely be discussing this further in a future version of the spec.

On the specific question of 3D spatial input, I'd encourage you to watch a short commentary from Edge Conf this year on this subject. [1] FWIW, I think several of us (myself included) have preordered a Leap Motion for experimentation with respect to Pointer Events. But I think the comments by the panelists at Edge Conf resonate with me very well. In particular,

1) The device landscape is pretty limited at the moment. Jumping to design an API tailored to just one or two retail devices will likely end up not being what we want in the long run and ultimately slow us down. 
2) We don't yet have a full picture of the use cases for web sites yet. If we wait a bit for the scenarios to bloom, then we can design an API that's better for the long run.
3) In line with #1 and #2, it's not clear what the right units for the web should be. The x and y coordinates map to screen coordinates. But depth (z) isn't a screen coordinate.  In LeapJS, for example, the units are millimeters for x, y, and z. For most scenarios we're targeting with Pointer Events, millimeters wouldn't be appropriate. Then also consider, how would we rationalize the design for other motion sensing devices like Xbox Kinect (which deals in a larger space) or the Samsung Galaxy S4's "Air View" (which deals in a closer space)?

While I think applying motion tracking devices to Pointer Events is an excellent idea, I think the ecosystem, devices, and use case development aren't yet mature enough to codify an API design into a web standard. So I think our previous decision to tackle extending PE to support additional device types in the V2 spec is still correct here.

>> If I'm not mistaken, I think this email to the list might constitute "raising an issue".  But I am not sure about the correct protocol to do that.  If anyone would care to let me know, please do.

Yes, this mail is all that's required to raise an issue. Thanks for contributing!  We can continue this discussion over mail and possibly during our next schedule teleconference (likely  26 March at 8am PST). [2]

-Jacob

[1] https://www.youtube.com/watch?v=zxuA-wyajhY&feature=player_detailpage#t=2454s (time: 40:45 - 46:00)
[2] http://www.w3.org/wiki/PointerEvents/Meetings

Received on Monday, 18 March 2013 19:09:56 UTC