- From: Rick Byers <rbyers@google.com>
- Date: Wed, 28 Jan 2015 12:11:22 -0500
- To: Olli Pettay <olli@pettay.fi>
- Cc: Alessandro Cogliati <a.cogliati@samsung.com>, Jacob Rossi <Jacob.Rossi@microsoft.com>, "public-touchevents@w3.org" <public-touchevents@w3.org>
- Message-ID: <CAFUtAY_AvWVgN1gn5R-190cO_YCgxvW0vkMc36TADp0To=TuLQ@mail.gmail.com>
On Wed, Jan 28, 2015 at 11:13 AM, Olli Pettay <olli@pettay.fi> wrote: > On 01/28/2015 06:01 PM, Rick Byers wrote: > >> Thanks Allesandro, I agree this is important. >> >> If it helps to move this discussion forward quicker, I'd be happy to take >> patches for such an API in chromium behind the "experimental web platform >> APIs" flag (we wouldn't ship them enabled by default until there is an >> official spec somewhere). There's some risk here, but as Denis's patch >> shows >> <https://codereview.chromium.org/750013004/> the implementation cost is >> low, so as long as discussion is happening in W3C and we have some doc >> somewhere defining the proposed API precisely, then I see no reason to >> block landing an experimental implementation in chromium. We can discuss >> further over on input-dev@chromium.org <mailto:input-dev@chromium.org> >> if you like. >> >> *Jacob, *I'd love to get your input on this. We all agreed in the >> conference call that there's increasing interest in stylus support. Even >> for those >> that feel PointerEvents is the best way forward, adding these properties >> to TEE is the only clear path to getting the PointerEvents polyfill to have >> stylus support in Chromium. >> > > > Still wondering... there really isn't any way to change PointerEvents spec > so that Google would be happy to implement it? > Making significant changes to Touch Events won't be in anyway more > backwards compatible than Pointer Events. > (I'm really worried that we'll end up having Touch Events and Touch > Events++ which won't be quite compatible, but will be hard to > feature detect the differences.) > There are specific things we could nitpick over, but as I've said these are all minor compared to the fragmentation question. Without a path to getting the API on all major browsers, I don't see how we would decide to ship PE in chromium (but as I've said before, we are keeping an open mind and watching the situation in the community - no decision is ever final). This discussion might be most productive on a call (happy to make it the focus of a whole call if folks are interested), but since we don't have one scheduled for 3 weeks let's see how far we can get over e-mail. The fundamental difference I see between these two approaches is about the level of risk due to the size of the surface area and changes required in applications. Eg. I don't see it as much of a problem that we have radiusX and radiusY on TouchEvent but no other browser (AFAIK) does (I've never heard someone complaint that that broke their website, or made it harder to write touch code that worked correctly across browsers). Safari doesn't support contact area, apps can feature detect for the API on browsers that do support it, and in most cases they don't need to use the API. Do you see our contact area support as creating a "TouchEvents++ which isn't quite compatible"? I see "tilt" support exactly the same as contact area - the risk for hurting platform fragmentation (at least in the mobile scenarios we're discussing) is relatively low. The key is that for the features both Chrome and Safari support, we have an identical API (modulo a couple minor issues we should work to fix). Developers should get "pay for play" - i.e. if they decide they want access to some additional feature (like tilt) they should be able to opt-into just that without it affecting other decisions/code in their system. Most developers won't need/care about tilt and so will pay zero cost. Any solution that forces a developer to pay a cost not directly related to the benefit they're trying to achieve is a non-incremental solution (eg. "to get tilt at this one single place, the input handling/routing logic in the app/framework must be extended to listen for pointer events"). Sometimes the net benefit is high enough to justify non-incremental solutions, but platform designers chronically over-estimate such benefits and are always disappointed by the slow adoption in the real-world where every development cost has to be justified on it's (often short-term) merits. Dart vs. TypeScript is a good example here where I've said Microsoft has a reasonable (incremental) path to success and Google doesn't. The hardest part of PE for us was that when we started down the path, we all agreed the net benefit of creating something new was worth it. By the time we had the freedom to make such a decision for ourselves (i.e. we forked blink), our priorities and the landscape had changed dramatically and it became apparent that the net benefit (compared to sticking with TE - despite it's problems) was no longer enough to justify the complexity cost in our developer story. Now if Safari came out with some different API for exposing stylus support, then I'd be concerned for exactly the fragmentation reasons you mention. As best as I've been able to determine, this is unlikely to happen. What do you mean "hard to feature detect the difference"? I agree that would be a disaster. We should be looking only at extensions to the base fully-interoperable API which are easy to feature detect. In this case "'tilt' in document.createTouch()" or something similar should be the perfect feature detection (as you've said elsewhere, feature detection is different from device capability detection - not talking about that here). > -Olli > > > >> Rick >> >> On Wed, Jan 28, 2015 at 6:28 AM, Alessandro Cogliati < >> a.cogliati@samsung.com <mailto:a.cogliati@samsung.com>> wrote: >> >> Hi Rick,____ >> >> Thank you Rick to share openly the community group thoughts. ____ >> >> I really appreciate.____ >> >> __ __ >> >> In general, we believe that the success of Web as platform depends on >> how it scales in different devices and how easy is for developers to create >> scalable Web applications (If they have to focus on a specific >> device, they go for native apps).____ >> >> I think Web reached good level of scalability about displaying the >> content (MediaQueries, etc…) but there is still work to do about Interaction >> models.____ >> >> __ __ >> >> We have followed Pointer Events specs, and we were ready to implement >> or optimized those in Chromium. But we also understand Chromium decision to >> not implement PE for all the reasons that have been mentioned.____ >> >> We don’t have strong opinion about that, PE or TEE, but we believe >> that new interaction models, e.g. Stylus, should be an extension of existing >> API and not new standalone specifications. ____ >> >> __ __ >> >> Denis’ patches go in that direction. We mapped PE features which we >> believe are important and that were missing in TEE, and implemented those >> in a >> way that are extensions of existing API.____ >> >> __ __ >> >> We are open to any discussion, and ready to contribute J____ >> >> ____ >> >> We can start also to think about touchscreen hover or other form of >> interactions…____ >> >> __ __ >> >> BR,____ >> >> Alessandro____ >> >> __ __ >> >> __ __ >> >> __ __ >> >> __ __ >> >> __ __ >> >> __ __ >> >> From: Rick Byers <rbyers@google.com >> <mailto:rbyers@google.com?Subject=Re%3A%202%20proposals% >> 20for%20stylus%20support.%20Extend%20range%20of%20% >> 20Touch.rotationAngle.%20Add%20Touch.tilt&In-Reply-To=% >> 3CCAFUtAY89MUessCqCMgict84%2BEHJpGyxBYTFaQtw1n%2BPDKgEH1g% >> 40mail.gmail.com%3E&References=%3CCAFUtAY89MUessCqCMgict84% >> 2BEHJpGyxBYTFaQtw1n%2BPDKgEH1g%40mail.gmail.com%3E>> >> Date: Tue, 27 Jan 2015 16:43:25 -0500 >> Message-ID: <CAFUtAY89MUessCqCMgict84+EHJpGyxBYTFaQtw1n+PDKgEH1g@ >> mail.gmail.com >> <mailto:CAFUtAY89MUessCqCMgict84%2BEHJpGyxBYTFaQtw1n% >> 2BPDKgEH1g@mail.gmail.com>> >> To: Denis Pikalov <d.pikalov@partner.samsung.com >> <mailto:d.pikalov@partner.samsung.com?Subject=Re%3A%202% >> 20proposals%20for%20stylus%20support.%20Extend%20range% >> 20of%20%20Touch.rotationAngle.%20Add%20Touch.tilt&In-Reply-To=% >> 3CCAFUtAY89MUessCqCMgict84%2BEHJpGyxBYTFaQtw1n%2BPDKgEH1g% >> 40mail.gmail.com%3E&References=%3CCAFUtAY89MUessCqCMgict84% >> 2BEHJpGyxBYTFaQtw1n%2BPDKgEH1g%40mail.gmail.com%3E>> >> Cc: Mustaq Ahmed <mustaq@google.com >> <mailto:mustaq@google.com?Subject=Re%3A%202%20proposals% >> 20for%20stylus%20support.%20Extend%20range%20of%20% >> 20Touch.rotationAngle.%20Add%20Touch.tilt&In-Reply-To=% >> 3CCAFUtAY89MUessCqCMgict84%2BEHJpGyxBYTFaQtw1n%2BPDKgEH1g% >> 40mail.gmail.com%3E&References=%3CCAFUtAY89MUessCqCMgict84% >> 2BEHJpGyxBYTFaQtw1n%2BPDKgEH1g%40mail.gmail.com%3E>>, >> "public-touchevents@w3.org >> <mailto:public-touchevents@w3.org?Subject=Re%3A%202% >> 20proposals%20for%20stylus%20support.%20Extend%20range% >> 20of%20%20Touch.rotationAngle.%20Add%20Touch.tilt&In-Reply-To=% >> 3CCAFUtAY89MUessCqCMgict84%2BEHJpGyxBYTFaQtw1n%2BPDKgEH1g% >> 40mail.gmail.com%3E&References=%3CCAFUtAY89MUessCqCMgict84% >> 2BEHJpGyxBYTFaQtw1n%2BPDKgEH1g%40mail.gmail.com%3E>" >> <public-touchevents@w3.org >> <mailto:public-touchevents@w3.org?Subject=Re%3A%202% >> 20proposals%20for%20stylus%20support.%20Extend%20range% >> 20of%20%20Touch.rotationAngle.%20Add%20Touch.tilt&In-Reply-To=% >> 3CCAFUtAY89MUessCqCMgict84%2BEHJpGyxBYTFaQtw1n%2BPDKgEH1g% >> 40mail.gmail.com%3E&References=%3CCAFUtAY89MUessCqCMgict84% >> 2BEHJpGyxBYTFaQtw1n%2BPDKgEH1g%40mail.gmail.com%3E>>____ >> >> __ __ >> >> Hi Denis,____ >> >> We discussed the high-level issue of stylus support in our conference >> call____ >> >> today <https://www.w3.org/2015/01/27-touchevents-minutes.html#item03>. >> We____ >> >> agree that stylus support is definitely something we need to solve >> for the____ >> >> platform. Pointer Events____ >> >> <https://dvcs.w3.org/hg/pointerevents/raw-file/tip/pointerEvents.html> >> is____ >> >> the only solution today, but that obviously doesn't help for browsers >> (like____ >> >> chromium) that have said they won't implement pointer events. >> We'll____ >> >> explore adding stylus-specific support to touch events, but it's >> going to____ >> >> take some time to come to a consensus in the group here (the whole TE >> vs.____ >> >> PE issue is very controversial). Please be patient with us as we try >> to____ >> >> come to an agreement within the group :-)____ >> >> __ __ >> >> Thanks,____ >> >> Rick____ >> >> __ __ >> >> __ __ >> >> On Tue, Jan 27, 2015 at 9:50 AM, Rick Byers <rbyers@google.com >> <mailto:rbyers@google.com?Subject=Re%3A%202%20proposals% >> 20for%20stylus%20support.%20Extend%20range%20of%20% >> 20Touch.rotationAngle.%20Add%20Touch.tilt&In-Reply-To=% >> 3CCAFUtAY89MUessCqCMgict84%2BEHJpGyxBYTFaQtw1n%2BPDKgEH1g% >> 40mail.gmail.com%3E&References=%3CCAFUtAY89MUessCqCMgict84% >> 2BEHJpGyxBYTFaQtw1n%2BPDKgEH1g%40mail.gmail.com%3E>> wrote:____ >> >> __ __ >> >> > On Tue, Jan 27, 2015 at 8:32 AM, Denis Pikalov <____ >> >> >d.pikalov@partner.samsung.com <mailto:d.pikalov@partner.samsung.com >> ?Subject=Re%3A%202%20proposals%20for%20stylus% >> 20support.%20Extend%20range%20of%20%20Touch.rotationAngle. >> %20Add%20Touch.tilt&In-Reply-To=%3CCAFUtAY89MUessCqCMgict84% >> 2BEHJpGyxBYTFaQtw1n%2BPDKgEH1g%40mail.gmail.com%3E&References=% >> 3CCAFUtAY89MUessCqCMgict84%2BEHJpGyxBYTFaQtw1n%2BPDKgEH1g% >> 40mail.gmail.com%3E>> wrote:____ >> >> >__ __ >> >> >> Thanks for yours comments____ >> >> >>__ __ >> >> >> @Rick:____ >> >> >> About Touch.rotationAngle. Right, rotationAngle is meaningless >> when____ >> >> >> tilt=0. Here are formulas for TEE/PE transitions (if rotationAngle >> defined____ >> >> >> as CW angle away from 0Y axis), range of rotationAngle [0..360), >> range of____ >> >> >> tilt [0..90]:____ >> >> >>__ __ >> >> >> TEE to PE:____ >> >> >> var tiltX = atan(tan(tilt) * sin(rotationAngle));____ >> >> >> var tiltY = atan(tan(tilt) * cos(rotationAngle));____ >> >> >> PE to TEE:____ >> >> >> var a = tan(tiltX);____ >> >> >> var b = tan(tiltY);____ >> >> >> var tilt = atan(sqrt(a*a + b*b));____ >> >> >> var rotationAngle = (b ? atan(a/b) : 90) + (b < 0 ? 180 : (a < 0 >> ? 360____ >> >> >> : 0));____ >> >> >>__ __ >> >> >> If we go with 360-degrees rotationAngle, we also need to define >> basis for____ >> >> >> rotationAngle (zero angle), since TEE doesn't define this. To >> align it with____ >> >> >> Android and due to lack in Android API (see explanation below) I >> would____ >> >> >> propose to use axis 0Y as basis.____ >> >> >>__ __ >> >> >__ __ >> >> > Right. Personally I think this further weight behind Mustaq's >> argument____ >> >> > that we should use a new property instead of changing the >> definition of____ >> >> > rotationAngle.____ >> >> >__ __ >> >> > Lack in Android API: I can't find API in Android to test whether____ >> >> >> orientation supported or not, the API returns zero rotation-angle >> in both____ >> >> >> cases - when angle is really 0 and when it's unknown. If TEE uses >> different____ >> >> >> basis for rotationAngle, let's say 0X, browser should add 90 >> degrees to the____ >> >> >> angle, retrieved from Android API. But this means, we get >> rotationAngle =____ >> >> >> 90 degrees, even in case when it's actually unknown.____ >> >> >>__ __ >> >> >__ __ >> >> > Does "InputDevice.getMotionRange(AXIS_ORIENTATION).getRange() > 0" >> work____ >> >> > for the devices you're looking at? I believe that's the >> intention.____ >> >> >__ __ >> >> > About S4 and finger tilt detection. I have to check, this is good >> idea.____ >> >> >>__ __ >> >> >>__ __ >> >> >> @Mustaq:____ >> >> >> Technically {tiltX, tiltY} and {tilt, tiltDirection} are equal, >> but if____ >> >> >> we'll go with your proposal, I would prefer tilt+tiltDirection >> since, since____ >> >> >> for simple use-cases, this is more easy to use, as this is less >> depends on____ >> >> >> device orientation.____ >> >> >>__ __ >> >> >__ __ >> >> > Makes sense. Drawing apps generally use tilt and tiltDirection____ >> >> > independently (or may use one but not the other), right?____ >> >> >__ __ >> >> >__ __ >> >> >>__ __ >> >> >> --____ >> >> >> Denis____ >> >> >>__ __ >> >> >>__ __ >> >> >> 26/01/15 23:44, Mustaq Ahmed пишет:____ >> >> >>__ __ >> >> >> I think we talking about two orthogonal ideas here that should be >> kept____ >> >> >> isolated in the spec: (A) touch surface geometry and (B) device____ >> >> >> orientation in 3D. TouchEvent specifies A perfectly but silent >> about B____ >> >> >> (which is, btw, precise in the PointerEvent spec). I suggest >> /adding/____ >> >> >> separate fields in TE to support B, rather than relying on >> existing fields____ >> >> >> meant for A. The new fields could be either:____ >> >> >> - {tiltX, tiltY} as in PE, or____ >> >> >> - {tilt, tiltDirection}, similar to Denis's suggestion.____ >> >> >>__ __ >> >> >> If we extend the rotationAngle range from 90 to 360 degrees to >> support____ >> >> >> B, any given touch ellipse for A could be specified in four >> different ways.____ >> >> >> For example, the ellipse (radius_x=rx, radius_y=ry, angle=15) is >> equivalent____ >> >> >> to all of {(rx, ry, 15+180), (ry, rx, 15+90), (ry, rx, 15+270)}, >> all of of____ >> >> >> which would conform to the spec. This would potentially force >> extra work____ >> >> >> for normalization every time a TE is consumed.____ >> >> >>__ __ >> >> >> Note that Android MotionEvent covers both A and B but through >> a____ >> >> >> conditional definition of the angle: orientation has different >> meanings for____ >> >> >> stylus and non-stylus devices. Correct me if I am missing >> something here. I____ >> >> >> think such "reuse" of a field makes the spec harder to follow, and >> forces____ >> >> >> usage-time-checking. I don't see a clear benefit in this approach, >> other____ >> >> >> than saving a byte or two. Memory is cheap now-a-days, >> code-maintenance is____ >> >> >> not.____ >> >> >>__ __ >> >> >> My two cents.____ >> >> >>__ __ >> >> >> Mustaq____ >> >> >>__ __ >> >> >>__ __ >> >> >>__ __ >> >> >> On Fri, Jan 23, 2015 at 3:27 PM, Rick Byers <rbyers@google.com >> <mailto:rbyers@google.com?Subject=Re%3A%202%20proposals% >> 20for%20stylus%20support.%20Extend%20range%20of%20% >> 20Touch.rotationAngle.%20Add%20Touch.tilt&In-Reply-To=% >> 3CCAFUtAY89MUessCqCMgict84%2BEHJpGyxBYTFaQtw1n%2BPDKgEH1g% >> 40mail.gmail.com%3E&References=%3CCAFUtAY89MUessCqCMgict84% >> 2BEHJpGyxBYTFaQtw1n%2BPDKgEH1g%40mail.gmail.com%3E>> wrote:____ >> >> >>__ __ >> >> >>> Hi Denis,____ >> >> >>> Thanks for joining the discussion here! I'd love to have >> Samsung____ >> >> >>> involved in this group. Another related topic we should discuss >> some time____ >> >> >>> if you (or a colleague) is interested is how the touchscreen >> hover____ >> >> >>> capabilities of the S4 should be exposed to the web (we've got a >> simple____ >> >> >>> prototype implementation <http://crbug.com/418188> in chrome >> already____ >> >> >>> behind a flag)____ >> >> >>>__ __ >> >> >>> Some relevant context for others in the group: Android stylus >> users____ >> >> >>> expect applications / websites to respond to their stylus as they >> do for____ >> >> >>> touch by default (eg. dragging sideways with the stylus on the >> home screen____ >> >> >>> switches pages, just like it does for touch). Some Android apps >> light up____ >> >> >>> to treat stylus differently, but for the most part it's treated >> like____ >> >> >>> touch. For this reason, android browsers (Samsung's browser, >> Chrome and____ >> >> >>> Firefox are all that I've tested) send touch events for stylus >> input. Even____ >> >> >>> if we end up sending something new like pointer events in the >> future, we'd____ >> >> >>> still need to send touch events for compatibility for the >> foreseeable____ >> >> >>> future.____ >> >> >>>__ __ >> >> >>> See inline____ >> >> >>>__ __ >> >> >>> On Thu, Jan 22, 2015 at 8:53 AM, Denis Pikalov <____ >> >> >>>d.pikalov@partner.samsung.com <mailto:d.pikalov@partner. >> samsung.com?Subject=Re%3A%202%20proposals%20for%20stylus% >> 20support.%20Extend%20range%20of%20%20Touch.rotationAngle. >> %20Add%20Touch.tilt&In-Reply-To=%3CCAFUtAY89MUessCqCMgict84% >> 2BEHJpGyxBYTFaQtw1n%2BPDKgEH1g%40mail.gmail.com%3E&References=% >> 3CCAFUtAY89MUessCqCMgict84%2BEHJpGyxBYTFaQtw1n%2BPDKgEH1g% >> 40mail.gmail.com%3E>> wrote:____ >> >> >>>__ __ >> >> >>>> Hi,____ >> >> >>>>__ __ >> >> >>>> As mentionedhttp://crbug.com/393462 about SPen: "...It's >> reasonable____ >> >> >>>> for an application to treat stylus input slightly differently >> from touch____ >> >> >>>> input. Ideally all details in Android's MotionEvent would be >> available to____ >> >> >>>> the web application... we should consider trying to standardize >> some____ >> >> >>>> additional properties on TouchEvent for this..."____ >> >> >>>>__ __ >> >> >>>> We agree, and we have been working to enable some feature we >> consider____ >> >> >>>> important.____ >> >> >>>> Please let us know your opinions about proposals below, which >> may make____ >> >> >>>> sense for stylus-type pointers:____ >> >> >>>>__ __ >> >> >>>> 1. Extend range of Touch.rotationAngle up to 360 degrees (to >> support____ >> >> >>>> oriented pointers).____ >> >> >>>> TEE defines only 90 degrees range for rotationAngle - due to >> symmetry,____ >> >> >>>> this is enough to define orientation of touch-ellipse, but we >> think, it____ >> >> >>>> makes sense to extend the range up to 360 degrees - and reuse >> this property____ >> >> >>>> to report orientation of pointer itself, if supported.____ >> >> >>>> Currently, orientation supported by samsung spen, at least.____ >> >> >>>>__ __ >> >> >>>__ __ >> >> >>> I definitely support this. Even outside the stylus use case, >> it's not____ >> >> >>> unreasonable that some "touchscreen" devices would be able to >> accurately____ >> >> >>> determine finger rotation (eg. by using hand/finger detection >> above the____ >> >> >>> surface of the screen). I see no reason why the extension should >> be____ >> >> >>> limited to 90 degrees (but we should have a note saying that in >> practice____ >> >> >>> many devices won't be able to report the full range).____ >> >> >>>__ __ >> >> >>> 2. Add property Touch.tilt____ >> >> >>>> Tilt can be defined as angle (in range 0..90 degrees) of the >> stylus____ >> >> >>>> away from the perpendicular to the screen. Normal use-cases are >> - advanced____ >> >> >>>> drawing applications, likehttp://goo.gl/jYExOt. Hardware >> support –____ >> >> >>>> yes, at least Note4 (+spen) supports tilt currently.____ >> >> >>>>__ __ >> >> >>> Patch for Touch.tilt:http://crrev.com/750013004,____ >> >> >>>> Tilt API:____ >> >> >>>>http://developer.android.com/reference/android/view/ >> MotionEvent.html#AXIS_TILT <http://developer.android.. >> com/reference/android/view/MotionEvent.html#AXIS_TILT>____ >> >> >>>>__ __ >> >> >>>__ __ >> >> >>> If others in the group are OK with adding stylus-specific >> properties____ >> >> >>> to touch events, then this sounds good to me. I guess in theory >> this could____ >> >> >>> represent the position of a finger as well (again with hardware >> that can____ >> >> >>> see / sense the finger above the surface), but I'm not sure >> anyone is doing____ >> >> >>> that in practice. Perhaps the S4 technically has this >> ability?____ >> >> >>>__ __ >> >> >>> I'd like to make sure we define these in a way that's easy to >> map____ >> >> >>> to/from the definitions in the PointerEvents spec____ >> >> >>> <https://dvcs.w3.org/hg/pointerevents/raw-file/tip/ >> pointerEvents.html#pointerevent-interface>,____ >> >> >>> so that implementations of pointer event polyfills can have full >> fidelity____ >> >> >>> (and libraries/frameworks can use whichever API they find more____ >> >> >>> convenient). There they use 180-degree tiltX and tiltY. >> 90-degree tilt +____ >> >> >>> 360 degree rotation as you've requested here should be easy to >> map to/from____ >> >> >>> that definition, right? In your definition, I assume stylus >> rotation is____ >> >> >>> meaningless when tilt=0, right? If we add this, then we should >> probably____ >> >> >>> include a note with the necessary formula to map between the >> two____ >> >> >>> representations.____ >> >> >>>__ __ >> >> >>> --____ >> >> >>>> Denis____ >> >> >>>>__ __ >> >> >>>>__ __ >> >> >>>__ __ >> >> >>__ __ >> >> >>__ __ >> >> >__ __ >> >> __ __ >> >> >> >
Received on Wednesday, 28 January 2015 17:12:14 UTC