Request for Comments: Proposal for Touch-Based Animation Scrubbing

I've been shopping this idea around in hallway conversation at the
last two meetings, and it seems that other browsers don't think I'm
crazy, so I've got a new proposal to put forth to all of you, for an
extension to the way animations work.

(This proposal is also located at
https://docs.google.com/document/d/1vRUo_g1il-evZs975eNzGPOuJS7H5UBxs-iZmXHux48/edit
if you want it in a more readable form, with worked-out examples.  No
need for a Google account to view.  If the proposal evolves during
discussion, that doc will also contain the most up-to-date version of
the proposal.)

Warning, this email is long.  If you want the gist, just read the
intro or Problem section, and then the suggested new properties.  To
save on further length, I've removed the worked-out example code from
this email - you can find it at the Google Doc above.

Touch-Based Animation Scrubbing
===============================
Or, Scrolling Through Time

This document introduces a set of CSS properties allowing authors to
declaratively bind CSS animations to touch and scroll gestures,
achieving a variety of common and useful effects.  This approach
should allow a common set of interaction designs to be handled very
easily, where today they require non-trivial JS, and should allow them
to be done purely on the compositor thread, for minimum jank.  Even
for animations that cannot be done purely on the compositor, these
properties allow for the possibility of heavier optimization, as they
avoid hanging the animation’s progress on the JS thread.

The Problem
-----------

We can do CSS animations faster than JS animations, because there’s no
need to invoke JS.  For a certain subset of CSS animations (ones that
only animate a subset of “compositor-friendly” properties), we can do
them *much* faster, and completely off the main thread, so JS-caused
jank is completely eliminated.

However, CSS animations are more-or-less static.  You define them
up-front with their triggers, and they simply happen, with no ability
to control them beyond pausing/removing them.  As soon as you want to
do something animation-like that is tied to user interaction, you’re
screwed - you’re forced back into JS animations (properties updated in
a rAF loop), which incurs an inherent and unavoidable chunk of delay,
and is subject to UI thread jank.  This makes it difficult to create
good touch-based interactions, particularly on mobile where JS-jank is
much more common.

Prior Art
---------

touch-based image carousels. Some of these are a finite stream, others
cycle back to the beginning.  Sometimes these want to “drift” into
showing the current image well.
the Android pull-down notifications drawer
the clank tab-list, where you can shift the tab list up and down,
revealing or hiding individual tabs, or flick tabs sideways, causing
them to curve downward and fade away
“zoom-based” UIs, where scrolling down instead zooms the view:
http://2011.beercamp.com/
scroll-based animations, where pictures move around, reveal
themselves, and otherwise change based on your current scroll
position: http://www.milwaukeepolicenews.com/ and many, many others
Twitter’s “pull-to-refresh” action on their mobile app, where
attempting to overscroll past the top reveals some elements and
triggers an XHR.


Additions to Animations
-----------------------


animation-timeline: global | <string>;

Add a new ‘animation-timeline’ property.  It accepts “global”, giving
the current normal behavior of animations, or a <string> indicating a
named timeline the animation will tie itself to.

The concept of named timelines is a fairly general one.  This proposal
will control the timeline declaratively, but there’s obvious
possibilities of, for example, creating the timeline as a manipulable
JS object, where scrubbing can be done directly in JS.

New Scrubbing Properties
------------------------

These names are very much tentative.  As well, it may end up being
easier to build this functionality directly into scrolling/overflow
behavior, as it hijacks scrolling for the element, rather than
reinventing several concepts on our own.

This is also a first draft of these properties, so the particular way
I’m laying out the syntax shouldn’t be considered too important right
now.  The general idea is what should be critiqued.

timeline-gesture: [ <gesture> && <string> ]#
timeline-length:  [
                    <time> ||
                    [ <length> | <percentage> ] ||
                    [ <integer> | infinite ]
                  ]#
timeline-momentum: [ none | momentum ]#;
timeline-notches: [ none | [ force | bias ] [ <time> | <percentage> ]+ ]#
<gesture> = scroll-up/down/left/right, scroll-vertical/horizontal,
overscroll-up/down/left/right, pinch-in/out, more?

‘timeline-gesture’ provides a list of timeline names and the gestures
that manipulate them.  The same timeline name can be provided for
multiple gestures in this property, or across multiple elements - it
just means that the same timeline can be scrubbed in multiple ways.

The scroll-up/down/left/right gestures can’t scrub a timeline into
negatives, only 0+.  Once you start a scroll-up gesture in one
direction, the other direction won’t activate until the first scrubs
all the way back to zero.  That is, if you scroll down 200px, then up
300px, the effect is to scrub the scroll-down timeline by 200px, then
back to 0, then scrub the scroll-up timeline by 100px.

Scroll-vertical/horizontal can scrub a timeline into negatives.

The overscroll-* gestures only scrub when you try to scroll “past the
edge” of a scrollable container.

The gestures, in general, are meant to be multi-modal.  For example,
you should be able to trigger the scroll gestures through touch
scrolling, scrollbar scrolling, mousewheel scrolling, keyboard
navigation, whatever.

‘timeline-length’ sets up the conversion between scrolling distance
and time, and how long the timeline is.  The length defaults to
“100%”, which is the width/height of the element, and the time
defaults to 1s.  The integer provided afterward specifies how many
multiples of the length can be scrubbed before the timeline stops,
defaulting to 1.

‘timeline-momentum’ controls whether, once you release the touch
gesture, the timeline continues to drift or not.  Possibly this can be
extended to specify the momentum curve.

‘timeline-notches’ establishes “notches” in the timeline, spots that
have special behavior when you stop scrubbing.  The behavior of the
notches is determined by the optional keyword at the start of the
property: “bias” will adjust the momentum curve, if a scrub would land
near a notch, to instead land exactly on the notch; “force” will cause
a timeline to automatically drift back to the nearest notch when you
stop actively scrubbing.

We should probably add an animation event that fires when you hit a
notch.  The notches will often correspond to important actions - for
example, the 0% and 100% notches in the mobile Chrome tab-fling
gesture indicate that the tab is closed - and authors will want
reliable, easy ways to respond to them, since the actual end of the
action won’t be easily observable (due to momentum and such).

Known Issues
------------

* Most of the time you’ll want the animation-timing-function to be
“linear”, so that the animation tracks the finger well, but this isn’t
the default.  Not much to do about this, I think.
* Take the “200px down, 300px up” example from the description of
‘timeline-gesture’.  What happens if a scroll-down gesture isn’t
registered at all?  Does scrolling down do nothing, then all 300px of
the upward scrolling goes to the scroll-up gesture?  Or do we continue
to track the total offset?  (I lean toward the former.)
* If an element and a child have the same (or compatible) gestures
registered, which wins?  Innermost or outermost?
* Using 'timeline-length' to specify a conversion ratio between
distance and time is a hack, but it's the shortest-path way to hook
this into the current animation syntax.  Are there better things we
can do?  Perhaps let 'animation-duration' accept a <string> to refer
to a timeline?  However, we should still allow an animation to only
occupy a fraction of a timeline, to make the "scrolling page
animations" thing easier to set up and maintain.  What I currently
have may be the best solution short of a larger re-architecting of the
way animations work.
* Some use-cases are annoyingly manual to set up.  For example, the
image carousel requires you to know ahead of time the size of each
item and how many there are, and spread this knowledge across a few
places.  If the items aren’t all the same size, or if there aren’t
enough of them to allow you to do a “safe” general wrap-around, you’ll
have to create individual @keyframes rules for each item.  I’m not
sure how to fix this without a more specialized solution that caters
to this specific use-case (which might be appropriate to build on top
of this).
* What happens if an element driving a vertical timeline has children
which can be vertically scrolled?  Do the children win when I start a
drag/mousewheel in them?  What if I’m using a mousewheel to scroll the
child and hit its bottom - should the scrolling “spill out” and turn
into scrubbing on the parent?  Should this be controllable?  What’s
the interaction of this and the overscroll gestures?
* We need an animation event that fires when you hit a notch.  The
notches will often correspond to important actions (examples: the 0%
and 100% notches in the mobile Chrome tab-fling gesture indicate that
the tab is closed; the 100% notch of the Twitter "pull to refresh"
animation should trigger an XHR) and authors will want reliable, easy
ways to respond to them, since the actual end of the action won’t be
easily observable (due to momentum and such).


Alternate Syntax Ideas
----------------------

Right now, if multiple gestures have the same timeline name, it’s
because they’re referring to the same timeline.  They can all scrub
the same timeline.  It seems potentially useful to have a “local”
timeline that only applies to your children, so that you don’t have to
invent unique names for every instance of a component on your page.
Maybe create ‘timeline-scope’ for creating a local timeline for your
subtree?

timeline-scope: none | <string>#

Now, if a ‘timeline-gesture’ or ‘animation-timeline’ property refers
to a named timeline, it walks up the tree looking for the first
ancestor with a ‘timeline-scope’ property with the given name.  If it
doesn’t find one, it assumes a global timeline of that name.






Thoughts?  Does anyone I haven't spoken to about this think it's
crazy?  Would you like to see working examples (done with a JS shim,
of course) of the functionality?  We haven't done any impl work yet,
but implementing this or something that solves the same problem is a
priority for the Chrome team in the next few months, so feedback
sooner rather than later would be great.

(If you're following the Web Animations work being done by some Moz
and WK engineers, they're aware of this and know how they'll slot it
into the general model.  It's pretty simple.)

~TJ

Received on Thursday, 29 November 2012 01:37:21 UTC