Browsers, Developers and Pointer Events Meeting Notes

Hello all,

The jQuery Foundation recently pulled together a meeting of representatives from IE, Chrome, Firefox, Dojo, IBM and the different jQuery project teams. We met to talk about Chrome’s decision to not implement Pointer Events, other browser thoughts and plans around Pointer Events and the future of Pointer Events from the point of view of library/framework maintainers and developers. Below are the notes from that meeting and they can also be viewed in this documenthttps://docs.google.com/document/d/17BUdF-uSg_0CUwhyenaB5FZeob0HK9DcD85lC1sH6cA

Thanks,
Kris Borchers
Executive Director - jQuery Foundation



Pointer Events Meeting Notes
2014-08-27, 2pm EDT

Attending:

Rick Byers, Google (Chrome)
Jacob Rossi, Microsoft (Internet Explorer)
Matt Brubeck, Mozilla (Firefox)
Daniel Freedman, Google (Polymer)
Scott Gonzalez, jQuery UI lead
Colin Snover, Dojo team
Dave Methvin, jQuery core lead
TJ VanToll, jQuery UI team, Telerik
Alex Schmitz, jQuery Mobile lead
Kris Borchers, Executive Director and jQuery UI team, jQuery Foundation
Jörn Zaefferer, jQuery UI team
Christophe Jolife, IBM
Andy Smith, IBM
Jacques Perrault, IBM

Introductions

Pointer Events (PE) implementation plans
	• Google Chrome
		• Will not implement
			• https://code.google.com/p/chromium/issues/detail?id=162757
			• http://lists.w3.org/Archives/Public/public-pointer-events/2014JulSep/0051.html
		• Feel it’s better for the web to extend touch events (since they’re more broadly supported) than to attempt to replace them.  Hope other browser vendors will work with us on extending touch events to address it’s limitations (blink doesn’t add APIs without support of other browsers).
		• Committed to improving the primitives, Rick Byers' team focused on improving input on the web (3 pillars: richness, rationality and performance)
		• Still want enabling technologies; example: touch-action CSS property
		• Feel it’s important that frameworks/apps aren’t constrained by consensus between browsers on “taste” questions like whether movement events should automatically capture or hit test every frame, or whether touch and mouse should dispatch events with the same name or different names. 
		• With appropriate primitives, libraries and frameworks (e.g., jQuery) can efficiently provide alternate API design choices  (also providing consistency across platforms where necessary)
	• Mozilla Firefox
		• Implementation contributed by MS Open Tech
		• Done for the Windows Metro Firefox, which has since been scrapped
		• Just started implementation for other platforms like FFOS, Android
		• Currently they work in (non-shipping) Metro, works in Firefox desktop for mouse input only, disabled by pref setting
	• Microsoft Internet Explorer
		• Implemented in IE11 (pre-standard version in IE10)
		• Supported on Windows Modern, desktop, and Windows Phone 8
		• Implemented Touch Events on mobile only for compat reasons
	• Apple Safari
		• (No representative present for official comment)
		• Publicly stated in Nov 2012 that they believe pointer events are a bad idea.  
		• Discussion at web input face-to-face reinforces this is still their belief

Scott: If Chrome expects web frameworks to use PE, wouldn't it make sense to implement PE in the browser? 

Rick: Yes, Chrome will revisit if it turns out most devs are using PE polyfills

Colin: What are the specific perf issues that you see in PE that are causing problems? In my work, performance is more of a red herring as the more important bit is an API that works everywhere. The TE improvements address the most egregious issues but it still isn't as elegant as PE.

Rick: No argument that PE is more elegant. If we had a path to universal input that all supported, we would be great with that, but not all browsers will support PE. If we had Apple on board with PE, we'd still be on board too. The equation has shifted for us.

Dave: Can you quantify the perf concern?

Rick: Hit testing cost, need a test on each PE move event. Fundamental performance cost, places new burdens on the performance of the system. Feels like the web is taking on a burden that the native mobile platforms don't have.  With better performance being the main reason developers site choosing native mobile platforms over the web, we’re unwilling to add any new architectural constraints that put the web at a performance disadvantage. Also a double dispatch issue, need to continue supporting touch and pointer for a decade or more.

Jacob: Double dispatch overhead only comes into play if both event models are used simultaneously, which isn't that likely

Rick: Lots of existing touch event code already exists. We think it's very unlikely that Apple will ever support Pointer Events.

Dave: But that means it's very hard to retrofit, Lots of code that makes bad assumptions like touch-only if they see TE, and it's hard to change the existing TE behavior (or add new pointer types) without breaking compat.

Jacob: A misconception about PE, you do often need to distinguish between touch and mouse models, but the advantage is that PE gives you this baseline pointer model that lets you define new pointer types.

Rick: On Android we (and Firefox) send touch events for stylus (e.g. Samsung) and mouse, usually it just works.

Jacob: Faking touch/pointer events gets to be hard.

Rick: We've done that though and usually it works.  We don’t consider it “fake” - trying to plan for a future where >90% of internet usage comes from touchscreen devices.  With touch primary, it’s not unreasonable to call other pointer input “touching” (just like enter sends ‘click’ events).  Maybe we have some compatible default for events, but provide a way to opt-in to getting them for other input devices?

Rick: … I don't think there are major performance issues with PE though.

Colin: It sounds like you're using perf as a reason to not to implement PE even though it's not a big deal

Rick: It's an input into the equation, but not a deciding factor. Fragmentation of the mobile web platform is the biggest concern.  If IE desktop supported TE we'd have a universal pointer model

Dave: But we know there's code that listens to touch but not mouse when it detects TE, which is broken on Chromebooks plus Chrome/IE on Windows.

Rick: We think we can push the web to fix by flipping TE on everywhere.

Jacob: From a 100 site survey these are some sites that were broken when both touch and mouse are on: ESPN.go.com, macys.com, onet.pl, pagesjuanes.fr,globo.com, webmd.com, samsung.com, taobao.com, adobe.com, huffingtonpost.com, etc.

Dave: The Chrome team has already had to back out several TE changes because they broke major sites, and jQuery has had similar experiences with its releases. Many sites just don't get updated very frequently, and forcing breakage won't encourage them to get fixed quickly

Rick: Anecdotally we believe this problem has gotten much better over the past 2 years.  Still a problem, but not nearly as big.  ESPN, for example, choosing to disable hover menus doesn’t appear to “break” the website - you can still navigate, just without a hover-oriented UI feature.  Would like feedback on TE extensions from others, in doing the work and looking at options it seemed like enhancements would be better. We're looking at successful native platforms and they are making design decisions similar to touch events. API that lets you opt in to sync scroll events for example.

Jacob: TE won't be interoperable based on Chrome+MS+Apple discussions, new extensions to TE won't work consistently across platforms.

Scott: What does the blink team envision as hover support in 5 years? Native? No support at all, or polyfills? 

Rick: I think it's native, there will be devices that support hover for a long time. It should be pay for play, if your device doesn't have hover you shouldn't have to pay for it. We think it's best to extend touch rather than mouse, given that we see usage shifting from web to native mobile every day.

Colin: If you think the web needs to be more rational in order to win, I'm not sure why TE will help given that we all think PE is more rational.

Rick: I think it's a false assumption that PE will be one interoperable model.  PointerEvents looks like it will ultimately lead to more fragmentation, not less.  We can’t discount Safari here - it’s very popular in the scenario we believe is most important: phones and tablets.

Colin: But won't we have 4 models now? TE, Google-enhanced TE, PE, mouse?

Dave: Which puts a further burden on frameworks and libs to rationalize them

Rick: We won't implement TE improvements if other vendors aren't on board

Scott: But if Apple won't implement some of them?

Rick: Perhaps Chrome+IE or Chrome+Firefox is enough

Colin: Why is Chrome+IE+Firefox not enough for PE, but Chrome+Firefox enough for TE extensions?

Dave: Also, you're asking for feedback from implementers, but here on this call you have several of the biggest users as well. Their input on PE/TE should be taken into account.

Rick: Agreed.  It’s precisely because there’s so much support for (and against) PE that this has been a long hard decision for us.  Input from jQuery, IE, etc. is very valuable to us.  I agree these 2 PE issues aren't sufficient alone to not do PE. Want richness as well, TE is richer and can do things that PE currently cannot (e.g., Pull to Refresh). 

Dave: Yes but some of that TE richness is at the cost of perf (e.g., preventDefault to opt out of scrolling). PE could easily be enhanced to add support for scenarios like Pull to Refresh, without backcompat concerns.

Jacob: TE isn't truly richer as these scenarios are being built with TE in ways that are not on par with native apps (e.g. perf).

Rick: These could be built with PE (or with minor changes to PE). The richness I'm talking about is the ability to handoff between JS event handlingand native scrolling within a single gesture. Native platforms have not made scrolling/event handling mutually exclusive.

Jacob: At the input F2F, even Apple agreed these problems shouldn't be solved with synchronous JS APIs.

Rick: We're interested in supporting frameworks, I want to help enable the PE polyfill to be successful and encourage framework devs to use the model they like

TJ: Can someone provide the current status of the PE polyfill in the Polymer repo? The current "deprecated" banner is probably discouraging its use.

Dan: Couldn't make dispatching PE fast enough for dragging on a phone, especially when there are many levels deep in shadow DOM. Nobody is working on it. I can help this process along. I broke a few compat cases (e.g. instanceof check) to make it faster but those could be backed out.

Colin: Dojo Foundation can take it.

Scott: Needs a formal transfer from Google, also needs to be simplified as far as the external dependencies go since it's tied to Polymer right now.

Dan: (Details on repo) I can write down some notes as issues and reopen bugs that were closed at deprecation. Will talk to my boss to see if the polyfill code can be transferred to another group.

Dan: We moved off the shim b/c the ability to have the target being the original element helps, but shim is written to do a hit test even in capture case (need it for pointer over/out)

Rick: http://code.google.com/p/chromium/issues/detail?id=398920 hit test perf

Scott: Sounds like that with shadow dom there is a big diff in hit testing 

Dan: Yes (8 levels takes 8 ms). getElementFromPoint is slow and cost scaled non-linearly with the number of levels of ShadowDOM. A hit test cache is the fix and Chrome is implementing one. Multiple gEFP calls in a tight loop only would need one expensive calc.

Rick B: If you want to see Chrome revisit the decision to implement Pointer Events, star https://code.google.com/p/chromium/issues/detail?id=162757