Re: Impact of pointer capture on hit testing requirements / performance

I just wanted to update the list on this discussion.  We've had a
discussion with Jacob offline which I'd summarize as:

1) The blink team has been working very hard to get as much done as
possible in our 16ms frame budget on finger move (especially on low power
mobile devices).  In one typical scenario the hit test costs us around
0.4ms (2.5% of the frame budget) and this is big enough to be a concern.

2) Pointer Events requires a hit test on every move, although it can be
avoided if the developer is careful.  Eg. they must call setPointerCapture
AND avoid any pointerenter/leave handlers on the element with capture, and
also avoid pointerover/out handlers on the element and all its ancestors.
 We are concerned that in practice developers will rarely be this careful.

3) Pointer Events is designed to allow UI elements to be activated by
touching and lifting anywhere within their visible bounds, and combined
with some pressed effect, is what requires a hit test on every move.  Other
mobile platforms (Andoid View and Cocoa) are designed (like touch events)
not to require a hit test on every move, determining activation based on
the motion alone (how the finger has moved, not what's underneath it).
 We're concerned that this difference means pointer events would put the
web platform at a performance disadvantage relative to existing mobile
platforms.

4) Other potential browser features (like snap points) could make some of
the effects we're focused on much cheaper, but we believe there's a long
tail of important custom effects which cannot be effectively baked into the
browser.

This issue isn't itself a deal breaker for Chrome's adoption of pointer
events, but I do think it's a legitimate argument without any obvious way
to address it.

Note that this difference in UI activation philosophy is somewhat
fundamental.  Eg. should <button>s be activated by touching and dragging
within their bounds then lifting on browsers that support both input
models?  I think they need to choose to either be inconsistent with
application UI elements designed for the touch event model or those
designed for the pointer event model.  Again, existing native touch
platforms are quite self-consistent in this regard.

Rick


On Tue, Jan 28, 2014 at 10:54 AM, Rick Byers <rbyers@google.com> wrote:

> Oh, I'm a little surprised that IE sets :hover when touching on one
> element and dragging over another.  I guess this makes sense since it's
> consistent with mouse behavior.  For blink, touch has always followed a
> model where the user is interacting only with the element they first put
> their finger on (since that's baked into the implicit capture property of
> touch events), and so we've never set :hover in this way.  Your button
> scenario makes perfect sense though, thanks!
>
> Ok, I'll have to gather some more data and get back to you.  A single hit
> test is relatively quick for us, but when (say) dragging an element around
> with CSS 3D transforms trying to get rock-solid 60fps on a slow mobile
> device, a single hit test on each frame can apparently make a non-trivial
> contribution to overrunning our 16ms budget for the frame (although I
> haven't looked at the data myself yet).  It's still a small fraction, but
> we have LOTs of things that are a small fraction of the frame time which
> can add up to overrunning the time for the frame if we're not careful.
>
> I was trying to think of how to optimize the hit test for this case.  It's
> non-trivial (both inside of blink, or in Polymer JavaScript code) because
> there could be any number of unrelated elements overlapping the capturing
> element so the boundary isn't easily describable.  I think a better
> optimization would be to skip the hit test entirely when the capturing
> element has no pointerover, pointerout, pointerenter, or pointerleave
> handler and no :hover style.  For the cases I'm concerned with, I think
> this would typically be true anyway.
>
> Rick
>
>
> On Tue, Jan 28, 2014 at 12:42 AM, Jacob Rossi <Jacob.Rossi@microsoft.com>wrote:
>
>> On Fri, Jan 24, 2014 at 8:29 AM, Rick Byers <rbyers@google.com> wrote:
>> >
>> > Hi,
>> > Do we think it's important that we require UA's to fire
>> pointerover/pointerout when crossing the boundary of an element that
>> currently has capture for that pointer?
>>
>> Yes, I think this is an important behavior.  It's common for UI like
>> buttons to take capture, and over/out effects are often used to indicate
>> visually whether the button would activate on lift, for example.
>>
>> > I ask because this requirement means that there's no way for an
>> application to suppress the need for the UA to do hit testing on every
>> pointermove (although in theory a UA could implement a simpler hit-testing
>> algorithm in this case).  In our performance testing of Polymer, we've
>> found this additional hit test to have some measurable overhead relative to
>> using touch events directly, and it's causing some people to ask whether
>> it's really worth using pointer events in polymer as a result (!).
>> > I'd like to change the Polymer PointerEvent polyfill not to do hit
>> testing on pointermove at all when an element has capture - then a
>> carefully designed application/framework should be able to get essentially
>> the same performance out of using pointer events as using raw touch events.
>>  But doing that would violate the below text we agreed to add to the spec
>> :-)
>>
>> It's always faster to do less work. But in this case, having hit testing
>> helps tremendously in a lot of scenarios.  While, yes, touch event code can
>> use APIs like elementFromPoint() to simulate the behavior, there's no
>> replacement for other hit testing behaviors like :hover and it makes having
>> true event dispatch to the hit node cumbersome and perhaps slow. Assuming
>> UAs implement simpler hit-testing in the capture case, the fast path is
>> only one API call away (setPointerCapture) with the current pointer events
>> behavior should you not need the hit-testing.
>>
>> That said, I'd be interesting in seeing any perf data you might have
>> about the impact here. At least in IE, I would expect a single hit test to
>> typically be negligible (on the order of uS). It's possible for UAs to
>> shortcut the hit testing for a pointer when it is captured by just
>> considering the box of the capture node. We experimented with this in IE,
>> but found the performance wins of not hit-testing to be so small that it
>> didn't warrant the additional complexity to our codebase. But perhaps such
>> an optimization could be tried in Polymer. If not, then perhaps this is an
>> argument for why a native implementation of pointer events is needed (so
>> UAs can short-circuit the hit test with capture). :-)
>>
>>
>