Re: 2 proposals for stylus support. Extend range of Touch.rotationAngle. Add Touch.tilt

Thanks Allesandro, I agree this is important.

If it helps to move this discussion forward quicker, I'd be happy to take
patches for such an API in chromium behind the "experimental web platform
APIs" flag (we wouldn't ship them enabled by default until there is an
official spec somewhere).  There's some risk here, but as Denis's patch
shows <https://codereview.chromium.org/750013004/> the implementation cost
is low, so as long as discussion is happening in W3C and we have some doc
somewhere defining the proposed API precisely, then I see no reason to
block landing an experimental implementation in chromium.  We can discuss
further over on input-dev@chromium.org if you like.

*Jacob, *I'd love to get your input on this.  We all agreed in the
conference call that there's increasing interest in stylus support.  Even
for those that feel PointerEvents is the best way forward, adding these
properties to TEE is the only clear path to getting the PointerEvents
polyfill to have stylus support in Chromium.

Rick

On Wed, Jan 28, 2015 at 6:28 AM, Alessandro Cogliati <a.cogliati@samsung.com
> wrote:

>  Hi Rick,
>
> Thank you Rick to share openly the community group thoughts.
>
> I really appreciate.
>
>
>
> In general, we believe that the success of Web as platform depends on how
> it scales in different devices and how easy is for developers to create
> scalable Web applications (If they have to focus on a specific device, they
> go for native apps).
>
> I think Web reached good level of scalability about displaying the content
> (MediaQueries, etc…) but there is still work to do about Interaction models.
>
>
>
> We have followed Pointer Events specs, and we were ready to implement or
> optimized those in Chromium. But we also understand Chromium decision to
> not implement PE for all the reasons that have been mentioned.
>
> We don’t have strong opinion about that, PE or TEE, but we believe that
> new interaction models, e.g. Stylus, should be an extension of existing API
> and not new standalone specifications.
>
>
>
> Denis’ patches go in that direction. We mapped PE features which we
> believe are important and that were missing in TEE, and implemented those
> in a way that are extensions of existing API.
>
>
>
> We are open to any discussion, and ready to contribute J
>
>
>
> We can start also to think about touchscreen hover or other form of interactions…
>
>
>
> BR,
>
> Alessandro
>
>
>
>
>
>
>
>
>
>
>
>
> From: Rick Byers <rbyers@google.com
> <rbyers@google.com?Subject=Re%3A%202%20proposals%20for%20stylus%20support.%20Extend%20range%20of%20%20Touch.rotationAngle.%20Add%20Touch.tilt&In-Reply-To=%3CCAFUtAY89MUessCqCMgict84%2BEHJpGyxBYTFaQtw1n%2BPDKgEH1g%40mail.gmail.com%3E&References=%3CCAFUtAY89MUessCqCMgict84%2BEHJpGyxBYTFaQtw1n%2BPDKgEH1g%40mail.gmail.com%3E>
> >
> Date: Tue, 27 Jan 2015 16:43:25 -0500
> Message-ID: <
> CAFUtAY89MUessCqCMgict84+EHJpGyxBYTFaQtw1n+PDKgEH1g@mail.gmail.com>
> To: Denis Pikalov <d.pikalov@partner.samsung.com
> <d.pikalov@partner.samsung.com?Subject=Re%3A%202%20proposals%20for%20stylus%20support.%20Extend%20range%20of%20%20Touch.rotationAngle.%20Add%20Touch.tilt&In-Reply-To=%3CCAFUtAY89MUessCqCMgict84%2BEHJpGyxBYTFaQtw1n%2BPDKgEH1g%40mail.gmail.com%3E&References=%3CCAFUtAY89MUessCqCMgict84%2BEHJpGyxBYTFaQtw1n%2BPDKgEH1g%40mail.gmail.com%3E>
> >
> Cc: Mustaq Ahmed <mustaq@google.com
> <mustaq@google.com?Subject=Re%3A%202%20proposals%20for%20stylus%20support.%20Extend%20range%20of%20%20Touch.rotationAngle.%20Add%20Touch.tilt&In-Reply-To=%3CCAFUtAY89MUessCqCMgict84%2BEHJpGyxBYTFaQtw1n%2BPDKgEH1g%40mail.gmail.com%3E&References=%3CCAFUtAY89MUessCqCMgict84%2BEHJpGyxBYTFaQtw1n%2BPDKgEH1g%40mail.gmail.com%3E>>,
> "public-touchevents@w3.org
> <public-touchevents@w3.org?Subject=Re%3A%202%20proposals%20for%20stylus%20support.%20Extend%20range%20of%20%20Touch.rotationAngle.%20Add%20Touch.tilt&In-Reply-To=%3CCAFUtAY89MUessCqCMgict84%2BEHJpGyxBYTFaQtw1n%2BPDKgEH1g%40mail.gmail.com%3E&References=%3CCAFUtAY89MUessCqCMgict84%2BEHJpGyxBYTFaQtw1n%2BPDKgEH1g%40mail.gmail.com%3E>"
> <public-touchevents@w3.org
> <public-touchevents@w3.org?Subject=Re%3A%202%20proposals%20for%20stylus%20support.%20Extend%20range%20of%20%20Touch.rotationAngle.%20Add%20Touch.tilt&In-Reply-To=%3CCAFUtAY89MUessCqCMgict84%2BEHJpGyxBYTFaQtw1n%2BPDKgEH1g%40mail.gmail.com%3E&References=%3CCAFUtAY89MUessCqCMgict84%2BEHJpGyxBYTFaQtw1n%2BPDKgEH1g%40mail.gmail.com%3E>
> >
>
>
>
> Hi Denis,
>
> We discussed the high-level issue of stylus support in our conference call
>
> today <https://www.w3.org/2015/01/27-touchevents-minutes.html#item03>.  We
>
> agree that stylus support is definitely something we need to solve for the
>
> platform.  Pointer Events
>
> <https://dvcs.w3.org/hg/pointerevents/raw-file/tip/pointerEvents.html> is
>
> the only solution today, but that obviously doesn't help for browsers (like
>
> chromium) that have said they won't implement pointer events.  We'll
>
> explore adding stylus-specific support to touch events, but it's going to
>
> take some time to come to a consensus in the group here (the whole TE vs.
>
> PE issue is very controversial).  Please be patient with us as we try to
>
> come to an agreement within the group :-)
>
>
>
> Thanks,
>
>    Rick
>
>
>
>
>
> On Tue, Jan 27, 2015 at 9:50 AM, Rick Byers <rbyers@google.com <rbyers@google.com?Subject=Re%3A%202%20proposals%20for%20stylus%20support.%20Extend%20range%20of%20%20Touch.rotationAngle.%20Add%20Touch.tilt&In-Reply-To=%3CCAFUtAY89MUessCqCMgict84%2BEHJpGyxBYTFaQtw1n%2BPDKgEH1g%40mail.gmail.com%3E&References=%3CCAFUtAY89MUessCqCMgict84%2BEHJpGyxBYTFaQtw1n%2BPDKgEH1g%40mail.gmail.com%3E>> wrote:
>
>
>
> > On Tue, Jan 27, 2015 at 8:32 AM, Denis Pikalov <
>
> > d.pikalov@partner.samsung.com <d.pikalov@partner.samsung.com?Subject=Re%3A%202%20proposals%20for%20stylus%20support.%20Extend%20range%20of%20%20Touch.rotationAngle.%20Add%20Touch.tilt&In-Reply-To=%3CCAFUtAY89MUessCqCMgict84%2BEHJpGyxBYTFaQtw1n%2BPDKgEH1g%40mail.gmail.com%3E&References=%3CCAFUtAY89MUessCqCMgict84%2BEHJpGyxBYTFaQtw1n%2BPDKgEH1g%40mail.gmail.com%3E>> wrote:
>
> >
>
> >>  Thanks for yours comments
>
> >>
>
> >> @Rick:
>
> >> About Touch.rotationAngle. Right, rotationAngle is meaningless when
>
> >> tilt=0. Here are formulas for TEE/PE transitions (if rotationAngle defined
>
> >> as CW angle away from 0Y axis),  range of rotationAngle [0..360), range of
>
> >> tilt [0..90]:
>
> >>
>
> >> TEE to PE:
>
> >>   var tiltX = atan(tan(tilt) * sin(rotationAngle));
>
> >>   var tiltY = atan(tan(tilt) * cos(rotationAngle));
>
> >> PE to TEE:
>
> >>   var a = tan(tiltX);
>
> >>   var b = tan(tiltY);
>
> >>   var tilt = atan(sqrt(a*a + b*b));
>
> >>   var rotationAngle = (b ? atan(a/b) : 90) + (b < 0 ? 180 : (a < 0 ? 360
>
> >> : 0));
>
> >>
>
> >> If we go with 360-degrees rotationAngle, we also need to define basis for
>
> >> rotationAngle (zero angle), since TEE doesn't define this. To align it with
>
> >> Android and due to lack in Android API (see explanation below) I would
>
> >> propose to use axis 0Y as basis.
>
> >>
>
> >
>
> > Right.  Personally I think this further weight behind Mustaq's argument
>
> > that we should use a new property instead of changing the definition of
>
> > rotationAngle.
>
> >
>
> > Lack in Android API: I can't find API in Android to test whether
>
> >> orientation supported or not,  the API returns zero rotation-angle in both
>
> >> cases - when angle is really 0 and when it's unknown. If TEE uses different
>
> >> basis for rotationAngle, let's say 0X, browser should add 90 degrees to the
>
> >> angle, retrieved from Android API. But this means, we get rotationAngle =
>
> >> 90 degrees, even  in case when it's actually unknown.
>
> >>
>
> >
>
> > Does "InputDevice.getMotionRange(AXIS_ORIENTATION).getRange() > 0" work
>
> > for the devices you're looking at?  I believe that's the intention.
>
> >
>
> > About S4 and finger tilt detection. I have to check, this is good idea.
>
> >>
>
> >>
>
> >> @Mustaq:
>
> >> Technically {tiltX, tiltY} and {tilt, tiltDirection} are equal, but if
>
> >> we'll go with your proposal, I would prefer tilt+tiltDirection since, since
>
> >> for simple use-cases, this is more easy to use, as this is less depends on
>
> >> device orientation.
>
> >>
>
> >
>
> > Makes sense.  Drawing apps generally use tilt and tiltDirection
>
> > independently (or may use one but not the other), right?
>
> >
>
> >
>
> >>
>
> >> --
>
> >> Denis
>
> >>
>
> >>
>
> >> 26/01/15 23:44, Mustaq Ahmed пишет:
>
> >>
>
> >> I think we talking about two orthogonal ideas here that should be kept
>
> >> isolated in the spec: (A) touch surface geometry and (B) device
>
> >> orientation in 3D. TouchEvent specifies A perfectly but silent about B
>
> >> (which is, btw, precise in the PointerEvent spec). I suggest /adding/
>
> >> separate fields in TE to support B, rather than relying on existing fields
>
> >> meant for A. The new fields could be either:
>
> >> - {tiltX, tiltY} as in PE, or
>
> >> - {tilt, tiltDirection}, similar to Denis's suggestion.
>
> >>
>
> >>  If we extend the rotationAngle range from 90 to 360 degrees to support
>
> >> B, any given touch ellipse for A could be specified in four different ways.
>
> >> For example, the ellipse (radius_x=rx, radius_y=ry, angle=15) is equivalent
>
> >> to all of {(rx, ry, 15+180), (ry, rx, 15+90), (ry, rx, 15+270)}, all of of
>
> >> which would conform to the spec. This would potentially force extra work
>
> >> for normalization every time a TE is consumed.
>
> >>
>
> >>  Note that Android MotionEvent covers both A and B but through a
>
> >> conditional definition of the angle: orientation has different meanings for
>
> >> stylus and non-stylus devices. Correct me if I am missing something here. I
>
> >> think such "reuse" of a field makes the spec harder to follow, and forces
>
> >> usage-time-checking. I don't see a clear benefit in this approach, other
>
> >> than saving a byte or two. Memory is cheap now-a-days, code-maintenance is
>
> >> not.
>
> >>
>
> >>  My two cents.
>
> >>
>
> >>  Mustaq
>
> >>
>
> >>
>
> >>
>
> >> On Fri, Jan 23, 2015 at 3:27 PM, Rick Byers <rbyers@google.com <rbyers@google.com?Subject=Re%3A%202%20proposals%20for%20stylus%20support.%20Extend%20range%20of%20%20Touch.rotationAngle.%20Add%20Touch.tilt&In-Reply-To=%3CCAFUtAY89MUessCqCMgict84%2BEHJpGyxBYTFaQtw1n%2BPDKgEH1g%40mail.gmail.com%3E&References=%3CCAFUtAY89MUessCqCMgict84%2BEHJpGyxBYTFaQtw1n%2BPDKgEH1g%40mail.gmail.com%3E>> wrote:
>
> >>
>
> >>> Hi Denis,
>
> >>> Thanks for joining the discussion here!  I'd love to have Samsung
>
> >>> involved in this group.  Another related topic we should discuss some time
>
> >>> if you (or a colleague) is interested is how the touchscreen hover
>
> >>> capabilities of the S4 should be exposed to the web (we've got a simple
>
> >>> prototype implementation <http://crbug.com/418188> in chrome already
>
> >>> behind a flag)
>
> >>>
>
> >>>  Some relevant context for others in the group: Android stylus users
>
> >>> expect applications / websites to respond to their stylus as they do for
>
> >>> touch by default (eg. dragging sideways with the stylus on the home screen
>
> >>> switches pages, just like it does for touch).  Some Android apps light up
>
> >>> to treat stylus differently, but for the most part it's treated like
>
> >>> touch.  For this reason, android browsers (Samsung's browser, Chrome and
>
> >>> Firefox are all that I've tested) send touch events for stylus input.  Even
>
> >>> if we end up sending something new like pointer events in the future, we'd
>
> >>> still need to send touch events for compatibility for the foreseeable
>
> >>> future.
>
> >>>
>
> >>>  See inline
>
> >>>
>
> >>> On Thu, Jan 22, 2015 at 8:53 AM, Denis Pikalov <
>
> >>> d.pikalov@partner.samsung.com <d.pikalov@partner.samsung.com?Subject=Re%3A%202%20proposals%20for%20stylus%20support.%20Extend%20range%20of%20%20Touch.rotationAngle.%20Add%20Touch.tilt&In-Reply-To=%3CCAFUtAY89MUessCqCMgict84%2BEHJpGyxBYTFaQtw1n%2BPDKgEH1g%40mail.gmail.com%3E&References=%3CCAFUtAY89MUessCqCMgict84%2BEHJpGyxBYTFaQtw1n%2BPDKgEH1g%40mail.gmail.com%3E>> wrote:
>
> >>>
>
> >>>>  Hi,
>
> >>>>
>
> >>>>  As mentioned http://crbug.com/393462 about SPen: "...It's reasonable
>
> >>>> for an application to treat stylus input slightly differently from touch
>
> >>>> input. Ideally all details in Android's MotionEvent  would be available to
>
> >>>> the web application... we should consider trying to standardize some
>
> >>>> additional properties on TouchEvent for this..."
>
> >>>>
>
> >>>> We agree, and we have been working to enable some feature we consider
>
> >>>> important.
>
> >>>> Please let us know your opinions about proposals below, which may make
>
> >>>> sense for stylus-type pointers:
>
> >>>>
>
> >>>> 1. Extend range of Touch.rotationAngle up to 360 degrees (to support
>
> >>>> oriented pointers).
>
> >>>> TEE defines only 90 degrees range for rotationAngle - due to symmetry,
>
> >>>> this is enough to define orientation of touch-ellipse, but we think, it
>
> >>>> makes sense to extend the range up to 360 degrees - and reuse this property
>
> >>>> to report orientation of pointer itself, if supported.
>
> >>>> Currently, orientation supported by samsung spen, at least.
>
> >>>>
>
> >>>
>
> >>>  I definitely support this.  Even outside the stylus use case, it's not
>
> >>> unreasonable that some "touchscreen" devices would be able to accurately
>
> >>> determine finger rotation (eg. by using hand/finger detection above the
>
> >>> surface of the screen).  I see no reason why the extension should be
>
> >>> limited to 90 degrees (but we should have a note saying that in practice
>
> >>> many devices won't be able to report the full range).
>
> >>>
>
> >>>   2. Add property Touch.tilt
>
> >>>> Tilt can be defined as angle (in range 0..90 degrees) of the stylus
>
> >>>> away from the perpendicular to the screen. Normal use-cases are - advanced
>
> >>>> drawing applications,  like http://goo.gl/jYExOt. Hardware support –
>
> >>>> yes, at least Note4 (+spen) supports tilt currently.
>
> >>>>
>
> >>>  Patch for Touch.tilt: http://crrev.com/750013004,
>
> >>>> Tilt API:
>
> >>>> http://developer.android.com/reference/android/view/MotionEvent.html#AXIS_TILT <http://developer.android..com/reference/android/view/MotionEvent.html#AXIS_TILT>
>
> >>>>
>
> >>>
>
> >>>  If others in the group are OK with adding stylus-specific properties
>
> >>> to touch events, then this sounds good to me.  I guess in theory this could
>
> >>> represent the position of a finger as well (again with hardware that can
>
> >>> see / sense the finger above the surface), but I'm not sure anyone is doing
>
> >>> that in practice.  Perhaps the S4 technically has this ability?
>
> >>>
>
> >>>  I'd like to make sure we define these in a way that's easy to map
>
> >>> to/from the definitions in the PointerEvents spec
>
> >>> <https://dvcs.w3.org/hg/pointerevents/raw-file/tip/pointerEvents.html#pointerevent-interface>,
>
> >>> so that implementations of pointer event polyfills can have full fidelity
>
> >>> (and libraries/frameworks can use whichever API they find more
>
> >>> convenient).  There they use 180-degree tiltX and tiltY.  90-degree tilt +
>
> >>> 360 degree rotation as you've requested here should be easy to map to/from
>
> >>> that definition, right?  In your definition, I assume stylus rotation is
>
> >>> meaningless when tilt=0, right?  If we add this, then we should probably
>
> >>> include a note with the necessary formula to map between the two
>
> >>> representations.
>
> >>>
>
> >>>   --
>
> >>>> Denis
>
> >>>>
>
> >>>>
>
> >>>
>
> >>
>
> >>
>
> >
>
>
>
>