Re: 2 proposals for stylus support. Extend range of Touch.rotationAngle. Add Touch.tilt

Hi Denis,
We discussed the high-level issue of stylus support in our conference call
today <https://www.w3.org/2015/01/27-touchevents-minutes.html#item03>.  We
agree that stylus support is definitely something we need to solve for the
platform.  Pointer Events
<https://dvcs.w3.org/hg/pointerevents/raw-file/tip/pointerEvents.html> is
the only solution today, but that obviously doesn't help for browsers (like
chromium) that have said they won't implement pointer events.  We'll
explore adding stylus-specific support to touch events, but it's going to
take some time to come to a consensus in the group here (the whole TE vs.
PE issue is very controversial).  Please be patient with us as we try to
come to an agreement within the group :-)

Thanks,
   Rick


On Tue, Jan 27, 2015 at 9:50 AM, Rick Byers <rbyers@google.com> wrote:

> On Tue, Jan 27, 2015 at 8:32 AM, Denis Pikalov <
> d.pikalov@partner.samsung.com> wrote:
>
>>  Thanks for yours comments
>>
>> @Rick:
>> About Touch.rotationAngle. Right, rotationAngle is meaningless when
>> tilt=0. Here are formulas for TEE/PE transitions (if rotationAngle defined
>> as CW angle away from 0Y axis),  range of rotationAngle [0..360), range of
>> tilt [0..90]:
>>
>> TEE to PE:
>>   var tiltX = atan(tan(tilt) * sin(rotationAngle));
>>   var tiltY = atan(tan(tilt) * cos(rotationAngle));
>> PE to TEE:
>>   var a = tan(tiltX);
>>   var b = tan(tiltY);
>>   var tilt = atan(sqrt(a*a + b*b));
>>   var rotationAngle = (b ? atan(a/b) : 90) + (b < 0 ? 180 : (a < 0 ? 360
>> : 0));
>>
>> If we go with 360-degrees rotationAngle, we also need to define basis for
>> rotationAngle (zero angle), since TEE doesn't define this. To align it with
>> Android and due to lack in Android API (see explanation below) I would
>> propose to use axis 0Y as basis.
>>
>
> Right.  Personally I think this further weight behind Mustaq's argument
> that we should use a new property instead of changing the definition of
> rotationAngle.
>
> Lack in Android API: I can't find API in Android to test whether
>> orientation supported or not,  the API returns zero rotation-angle in both
>> cases - when angle is really 0 and when it's unknown. If TEE uses different
>> basis for rotationAngle, let's say 0X, browser should add 90 degrees to the
>> angle, retrieved from Android API. But this means, we get rotationAngle =
>> 90 degrees, even  in case when it's actually unknown.
>>
>
> Does "InputDevice.getMotionRange(AXIS_ORIENTATION).getRange() > 0" work
> for the devices you're looking at?  I believe that's the intention.
>
> About S4 and finger tilt detection. I have to check, this is good idea.
>>
>>
>> @Mustaq:
>> Technically {tiltX, tiltY} and {tilt, tiltDirection} are equal, but if
>> we'll go with your proposal, I would prefer tilt+tiltDirection since, since
>> for simple use-cases, this is more easy to use, as this is less depends on
>> device orientation.
>>
>
> Makes sense.  Drawing apps generally use tilt and tiltDirection
> independently (or may use one but not the other), right?
>
>
>>
>> --
>> Denis
>>
>>
>> 26/01/15 23:44, Mustaq Ahmed пишет:
>>
>> I think we talking about two orthogonal ideas here that should be kept
>> isolated in the spec: (A) touch surface geometry and (B) device
>> orientation in 3D. TouchEvent specifies A perfectly but silent about B
>> (which is, btw, precise in the PointerEvent spec). I suggest /adding/
>> separate fields in TE to support B, rather than relying on existing fields
>> meant for A. The new fields could be either:
>> - {tiltX, tiltY} as in PE, or
>> - {tilt, tiltDirection}, similar to Denis's suggestion.
>>
>>  If we extend the rotationAngle range from 90 to 360 degrees to support
>> B, any given touch ellipse for A could be specified in four different ways.
>> For example, the ellipse (radius_x=rx, radius_y=ry, angle=15) is equivalent
>> to all of {(rx, ry, 15+180), (ry, rx, 15+90), (ry, rx, 15+270)}, all of of
>> which would conform to the spec. This would potentially force extra work
>> for normalization every time a TE is consumed.
>>
>>  Note that Android MotionEvent covers both A and B but through a
>> conditional definition of the angle: orientation has different meanings for
>> stylus and non-stylus devices. Correct me if I am missing something here. I
>> think such "reuse" of a field makes the spec harder to follow, and forces
>> usage-time-checking. I don't see a clear benefit in this approach, other
>> than saving a byte or two. Memory is cheap now-a-days, code-maintenance is
>> not.
>>
>>  My two cents.
>>
>>  Mustaq
>>
>>
>>
>> On Fri, Jan 23, 2015 at 3:27 PM, Rick Byers <rbyers@google.com> wrote:
>>
>>> Hi Denis,
>>> Thanks for joining the discussion here!  I'd love to have Samsung
>>> involved in this group.  Another related topic we should discuss some time
>>> if you (or a colleague) is interested is how the touchscreen hover
>>> capabilities of the S4 should be exposed to the web (we've got a simple
>>> prototype implementation <http://crbug.com/418188> in chrome already
>>> behind a flag)
>>>
>>>  Some relevant context for others in the group: Android stylus users
>>> expect applications / websites to respond to their stylus as they do for
>>> touch by default (eg. dragging sideways with the stylus on the home screen
>>> switches pages, just like it does for touch).  Some Android apps light up
>>> to treat stylus differently, but for the most part it's treated like
>>> touch.  For this reason, android browsers (Samsung's browser, Chrome and
>>> Firefox are all that I've tested) send touch events for stylus input.  Even
>>> if we end up sending something new like pointer events in the future, we'd
>>> still need to send touch events for compatibility for the foreseeable
>>> future.
>>>
>>>  See inline
>>>
>>> On Thu, Jan 22, 2015 at 8:53 AM, Denis Pikalov <
>>> d.pikalov@partner.samsung.com> wrote:
>>>
>>>>  Hi,
>>>>
>>>>  As mentioned http://crbug.com/393462 about SPen: "...It's reasonable
>>>> for an application to treat stylus input slightly differently from touch
>>>> input. Ideally all details in Android's MotionEvent  would be available to
>>>> the web application... we should consider trying to standardize some
>>>> additional properties on TouchEvent for this..."
>>>>
>>>> We agree, and we have been working to enable some feature we consider
>>>> important.
>>>> Please let us know your opinions about proposals below, which may make
>>>> sense for stylus-type pointers:
>>>>
>>>> 1. Extend range of Touch.rotationAngle up to 360 degrees (to support
>>>> oriented pointers).
>>>> TEE defines only 90 degrees range for rotationAngle - due to symmetry,
>>>> this is enough to define orientation of touch-ellipse, but we think, it
>>>> makes sense to extend the range up to 360 degrees - and reuse this property
>>>> to report orientation of pointer itself, if supported.
>>>> Currently, orientation supported by samsung spen, at least.
>>>>
>>>
>>>  I definitely support this.  Even outside the stylus use case, it's not
>>> unreasonable that some "touchscreen" devices would be able to accurately
>>> determine finger rotation (eg. by using hand/finger detection above the
>>> surface of the screen).  I see no reason why the extension should be
>>> limited to 90 degrees (but we should have a note saying that in practice
>>> many devices won't be able to report the full range).
>>>
>>>   2. Add property Touch.tilt
>>>> Tilt can be defined as angle (in range 0..90 degrees) of the stylus
>>>> away from the perpendicular to the screen. Normal use-cases are - advanced
>>>> drawing applications,  like http://goo.gl/jYExOt. Hardware support –
>>>> yes, at least Note4 (+spen) supports tilt currently.
>>>>
>>>  Patch for Touch.tilt: http://crrev.com/750013004,
>>>> Tilt API:
>>>> http://developer.android.com/reference/android/view/MotionEvent.html#AXIS_TILT
>>>>
>>>
>>>  If others in the group are OK with adding stylus-specific properties
>>> to touch events, then this sounds good to me.  I guess in theory this could
>>> represent the position of a finger as well (again with hardware that can
>>> see / sense the finger above the surface), but I'm not sure anyone is doing
>>> that in practice.  Perhaps the S4 technically has this ability?
>>>
>>>  I'd like to make sure we define these in a way that's easy to map
>>> to/from the definitions in the PointerEvents spec
>>> <https://dvcs.w3.org/hg/pointerevents/raw-file/tip/pointerEvents.html#pointerevent-interface>,
>>> so that implementations of pointer event polyfills can have full fidelity
>>> (and libraries/frameworks can use whichever API they find more
>>> convenient).  There they use 180-degree tiltX and tiltY.  90-degree tilt +
>>> 360 degree rotation as you've requested here should be easy to map to/from
>>> that definition, right?  In your definition, I assume stylus rotation is
>>> meaningless when tilt=0, right?  If we add this, then we should probably
>>> include a note with the necessary formula to map between the two
>>> representations.
>>>
>>>   --
>>>> Denis
>>>>
>>>>
>>>
>>
>>
>