- From: Dael Jackson <daelcss@gmail.com>
- Date: Sun, 27 Aug 2017 14:31:58 -0400
- To: public-houdini@w3.org
- Cc: www-style@w3.org
=================================================
These are the official Houdini Task Force
minutes. Unless you're correcting the
minutes, please respond by starting
a new thread with an appropriate subject line.
=================================================
AnimationWorklet
----------------
- flackr introduced the newest iteration on the spec which takes
into account previous feedback that it wasn't fitting well
into web animations. Spec here: https://wicg.github.io/animation-worklet/
- General response was very positive that this seemed like it
would solve use cases well and integrate with existing specs.
- There was conversation around exactly what data on user state
should be passed in; till will create a github issues to
continue this conversation.
- Detecting when a user is in a mutating state is another topic
that will be added to github for conversation by Rossen.
- Lastly, the spec will need to detail how to address when how to
handle when a specific timeline is requested, but the goal
isn't being met so timeline is being scaled down. Dino will
file an issue on this topic.
TypedOM
-------
- RESOLVED: StyleMap for CSS properties is ASCII case-insensitive
because that's how CSS property names are defined.
(Issue #309)
- RESOLVED: CSSURLImageValue that fails to parse as a URL should
throw an error. (Issue #424)
- RESOLVED: Publish TypedOM
-> https://www.w3.org/TR/2017/WD-css-typed-om-1-20170801/
===== FULL MINUTES BELOW ======
Agenda: https://github.com/w3c/css-houdini-drafts/wiki/Paris-F2F-August-1st-2017
Present:
Rossen Atanassov, Microsoft
Tab Atkins, Google
David Baron, Mozilla
Brian Birtles, Mozilla
Tantek Çelik, Mozilla
Emil A Eklund, Google
Elika Etemad, Invited Expert
Rob Flack, Google
Koji Ishii, Google
Dean Jackson, Apple
Ian Kilpatrick, Google
Peter Linss, Invited Expert/TAG
Jack Moffitt, Mozilla
Naina Raisinghani, Google
François REMY, Microsoft
Simon Sapin, Mozilla
Nicolas Silva, Mozilla
Alan Stearns, Adobe
Surma, Google
Sergio Villar, Igalia
Jet Villegas, Mozilla
Eric Willigers, Google
AnimationWorklet
================
Scribe: TabAtkins
<jack> https://wicg.github.io/animation-worklet/
flackr: A while back we presented a CSS version of AnimationWorklet.
flackr: Since then we've taken the concerns about this fitting
with Web Animations and turned it into a more imperative
API, like a custom Web Animation.
flackr: The construction takes the name of an animator, like the
PaintWorklet api, and an effect in the timeline.
flackr: Then the animator can do whatever it wants to manipulate
the effect.
flackr: This gives the same flexibility as Web Animation, letting
you manipulate multiple elements as a group effect.
flackr: This brings more benefits, too.
flackr: No longer have to worry about the final output value, this
is just part of the effects stack.
flackr: Now it can run between other main-thread animation
effects, run out of sync, etc.
<flackr> https://googlechrome.github.io/houdini-samples/animation-worklet/parallax-scrolling/?nojank
flackr: This parallax demo shows simplest example.
flackr: Construct an effect that can affect transformY and
transformX, read the scroll offset, and update the
transforms.
flackr: This can be done with just scrollTimeline and a regular
web animation, but with this you can do more complex
effects on top as well.
flackr: Because we're using a single input timeline, you can
explain how the animation calls finish/reverse/playback
rate affects the animations.
flackr: We allow in the spec passing extra timelines, and
attaching/detaching them.
flackr: Like an animation that invalidates on scroll, which then
attaches itself to the document timeline for a little
while before removing itself. A scroll-trigger animation,
rather than scroll-driven.
<flackr> https://googlechrome.github.io/houdini-samples/animation-worklet/spring-sticky/
flackr: This example invalidates on scroll, then attaches itself
to the document timeline to continue until it finishes,
then detaches.
flackr: Another benefit of this is that it's pretty easy to
polyfill. Doing the CSS version was very gross, had to
walk the tree.
flackr: Other than running on the main thread, the polyfill is
quite performant now.
iank: Nice thing about this model now is that there's conceptually
only one timeline we're listening to at a given moment, the
one the animation gets invalidated on.
iank: Could be scroll, could be document.
<iank> https://googlechrome.github.io/houdini-samples/animation-worklet/spring-sticky/spring-sticky-animator.js
[discussion of demo]
iank: So looking at demo, new WorkletAnimation gives the name of
the animator
iank: And an array of effects.
iank: Behind the scenes, we bump it up into a group effect.
iank: You get passed one timeline, the primary timeline; in this
case it's scroll.
iank: And you can pass in an options bag, it's structured cloned
over to the animation worklet.
iank: And inside the springStickyAnimator.js, there's a
registerAniamtor call.
iank: Its constructor takes the object bag, pulls out the
documentTimeline from it.
iank: It animates on the scroll timeline.
iank: Then it attaches and detaches from the documentTimeline to
make it invalidate properly (and finish up).
iank: To actually apply effect you set the localTime of each
individual effect that's passed in.
iank: This integrates nicely if we ever have sequential group
effects in web animations, you'll just set the localTime of
that.
iank: Because we're integrating the web animation timelines, play/
pause/etc just falls out.
iank: This example also has some internal state in the class.
iank: There's a velocity and position array.
iank: So in this case the springs can keep track of velocity
across invocations.
<astearns> I kind of understand the scroll down animation in the
second example. The little jump when you scroll up is
pretty weird, though.
<surma> astearns: That is most likely the polyfill in action. IIUC
a native implementation wouldn’t have that jitter
dino: When the animation detaches itself from document timeline,
that doesn't cause the animation worklet to go away, right?
flackr: Right, it's still attached to the scroll timeline.
dino: And what makes it attach again?
flackr: It's always attached to the scroll timeline.
flackr: In the HTML file you specify the scroll timeline as the
primary; you just attach to the document timeline manually
when the animation starts.
dino: What's the currentTime that's passed in?
iank: The main timeline, scroll in this case.
dino: So attaching to the document timeline just says, call me on
every document tick.
iank: Yeah.
iank: There's another way to do this - you can attach to the
documentTimeline as primary, then just read the
scrollTimeline on every tick. That's not as performant
though.
iank: This way, you can declare when you care about continuous
time.
iank: We could avoid the performance issues of passing in the
document timeline by waiting on animation triggers and
self-destroying, but that's more complicated.
dino: And the way the animations move is you set localTime on the
effect, which affects scroll position...
iank: Yes.
dino: I like that this is all done on web animations.
dino: We do our scrolling in a separate process.
dino: If possible we don't want to send client JS to that process;
exposes us to exploits.
dino: But the animation code still needs to run somewhere...
flackr: In Chrome I think we'll have a thread for the animation
worklets, and send it work - these timelines have new
values, run and get back the local times produced.
flackr: Don't think we'll run it in our scroll-handling thread.
iank: We also have deadlines, so slow script'll just not work.
iank: And we'll fall back to the main thread if animating a
non-accelerating property.
dino: Is sticky okay if running in a non-scrolling thread?
flackr: We haven't found it to be an issue. It seems to be
possible to pass back and forth state before running out
of time.
iank: We have a deadline of a few ms to let things finish; as long
as it hits that deadline we can keep it in sync.
dino: This all looks great; it kinda concerns me that - this is
more of a metacomment - we've spent 2.5 years speccing
low-level stuff without getting much, when we could have
done high-level stuff by now.
dino: Lots of code here to do parallax, could have been done very
simply previously.
dino: Worries me a bit how much complexity there is to do
relatively simple things. This sticky spring isn't something
we'd want to build in, obviously, but parallax could have
been easy.
birtles: I don't think that's quite the case. There's a piece
between, Scroll-Driven Animations, and parallax could
just build on that. It has both an API and a declarative
syntax.
dino: I'm saying we could have done less work just doing parallax.
birtles: I think this is trying to join dots - if the built-in
doesn't cover your use-case, can you extend it out.
birtles: I think this helps validate the scroll animation proposal.
iank: I think previous attempts, too, weren't as simple. Didn't
have an easy way to listen to scrolls, for example.
dino: My only concern is that I wish we could produce some logic I
could send to the scroll thread without having to send JS...
TabAtkins: I'm really happy with how general this is, because it
means we can delegate a lot of hard cases to it, and
then do Scroll-Linked Animations limited to the simpler
cases. It was way too difficult ticking all the boxes
with just SLA when I tried it in the past; there's a
nice simple core we can limit ourselves to now.
dino: That sounds good.
birtles: There are some details here I'm not sure about, like
blessing one timeline above others, but these are details
we can work on. I think the approach is good overall, and
I'm happy it's working on top of web animations.
birtles: I didn't realize the spec had been substantially updated,
so I'm still going over those issues.
dino: What do you think about implementability?
birtles: Dunno yet.
till: The internal state that has to be transferred to a worklet
means that it's not possible to ever parallelize this
further.
till: If instead we did something like provide a velocity of the
current scroll, or last location, we could potentially do
away with internal state, and speculatively execute multiple
steps in the timeline, and use them if necessary.
flackr: It's worth pointing out that you can run each animator
independently.
till: Understood.
flackr: We'd have to specify all different pieces of state ...
TabAtkins: Not necessarily all, just some useful stuff. With a
properly designed API, could detect when they're only
relying on pure state.
till: Or restrict them to only pure state, if that's not too
restrictive.
flackr: Another header example is where your action depends on
scroll direction in the past...
till: Right, we could pass that in.
TabAtkins: Or just, instead of mutating, always have it run on
pure state that it can produce new versions of.
flackr: Yeah, maybe your animation could specify that it's clean,
and we could have a policy of regularly disposing of
"clean" animations.
till: This isn't about tearing down an animation, it's about
running multiple frames interleaved, and reducing the risk
of getting jank in this one animation. That's only possible
with pure state.
TabAtkins: Yeah, this could easily be an array of pure position/
velocity state, and just returning an updated clone.
iank: A concern was the cost of structured cloning every frame.
Would be better if we could just clone every N frames.
flackr: This is coming from the world where devs would usually
write this as a main-thread effect, so here at least they
can be run separately.
till: Right, so I see that being able to keep internal state is
more powerful, but it keeps you from parallelizing.
flackr: I wonder if there's verification possibility - analyze the
script to see if it's pure enough.
till: No, we tried that with parallel.js
TabAtkins: And it's easy to fall out of the pit of success for no
apparent reason.
iank: So if we returned the state every frame from animate, and
passed it back in, we could do pure state.
till: Not quite - we want to ask for frame N, *and* start working
on frame N+1, without waiting for frame N to finish.
till: You could increase your frame budget from 16ms to
32ms if you're doing frames on 2 different cores
TabAtkins: Ah, so that does require only relying on UA-provided
state.
till: Yeah, letting us take care of animations on multiple cores.
surma: People are already asking for how to depend on user input.
That breaks speculation, right?
till: People don't usually change the velocity of their input in
short timescales (30ms or so). You'll occasionally
mis-speculate, but mostly will be okay.
birtles: And doing multiple frames at slight offsets to produce
motion blur.
TabAtkins: I think this is all quite tractable with minimal
changes.
iank: Would you be okay with starting with a mode that says you
only use pure state, and we'll tear down your object
regularly to prevent you from relying on impure state?
till: I probably can't convince you to actually produce fresh
objects on every frame, so yeah.
TabAtkins: Get all the JS engines to give us cheap immutable
state, and we'll talk. ^_^
till: And spec some additional pure state info the UA can provide
to the animator.
iank: Velocity, acceleration, maybe last N frames of scroll data...
iank: So thinking maybe we start with a separate mode, see what
people are using with it, start adding more data.
jack: Could we pass in velocity and acceleration by default? You
pretty much have to calculate that anyway.
till: Problem with the flag is that we can't default to that.
Maybe if we just speculate wrongly for the first N frames we
can switch to the stateful version?
flackr: We have this idea that we can add info to the timeline
that contains this extra info, like user input too.
iank: We could invert the flag - default to not passing in new
state, you can flag it on to get user-defined state passed
around.
till: Seems useful regardless of speculative execution.
till: If we always pass in these values, people can rely on it,
and we can just assume things are pure unless we fail
speculation.
<iank> https://gist.github.com/anonymous/e22250dbdfa92da59508a220b8087d9d
TabAtkins: How do you speculate if people *are* using mutating
state? Seems you'd mess up the state.
till: Run one thread as source of truth for a while, speculatively
run the same frames ahead of time on another thread. If
speculation fails for more than N out of the first M frames (
speculation gives different result than the "real" thread),
assume they're using mutating state and stop speculating.
Otherwise, continue speculating.
till: Something that could be helped by sequential speculative
execution is power usage - better to spike the CPU in bursts
rather than run it continuously.
dino: And with 120Hz animation, letting the animator idle as much
as possible would be good.
till: And with 4 cores, you can split frames between them, have
better guarantee of hitting every frame.
dino: So I think you're saying - based on what's happening now,
you'll be issuing a lot of updates in a short amount of
time, so you want to use other cores to pregen some frames
ahead now, so they're ready when needed.
dino: So the speculative ones return frames, timestamped, the
system uses them?
TabAtkins: Yeah. Speculate for the first N frames along with
primary thread, compare values; assume speculation is
safe if it succeeds.
dino: Why not let the worklet return multiple successive frames?
TabAtkins: That doesn't solve the problem, right? You're still
spending the same amount of CPU as calling it multiple
times.
dino: Like, if I know the animation wants to run at 120Hz, but
I'll be called at 60Hz, you can do two frames at once.
till: That still requires you to calculate frames at 120Hz. With
speculation you can actually pump frames at 60Hz, just every
other frame on each core.
TabAtkins: Probably useful to have a flag that specifies whether
you think you're safe to speculate or not. If you claim
you are, we do some things to keep you honest (tear
down your object regularly); if you claim you're not,
UAs might still do speculation speculatively to see if
you really are, as till described earlier.
flackr: This sounds sensible.
till: This is also based on impl experience we're having with
Paint Worklet, we're already doing speculation there.
Slightly different cases there, but similar API.
<till> PR for speculative paint worklet execution:
https://github.com/servo/servo/pull/17810
dino: Also would like a flag specifying how fast timeline should
pump. rAF is kinda 60Hz, but we now have variable-rate
screens coming out, etc.
dino: So question is how to specify that from a worklet - message
to the timeline saying "I wanna get called back more
frequently/in at least 4ms"? And some feedback saying you're
not meeting that goal, so you're getting scaled back.
flackr: Would you ever not want to run that fast?
dino: Yes, if your animation doesn't need to be 120hz, it's better
to run at 60hz for power reasons.
<brucel> +1 to what dino said "if your animation doesn't need to
be 120hz, it's better to run at 60hz for power reasons."
- Opera desktop, for example, has power saving mode in
which frame rate for video and animations is deliberately
reduced
TabAtkins: This kinda sounds like various stream APIs, which let
you attach to timelines at various coarseness.
dino: and figuring out how fast a timeline *can* be pumped, asking
for a particular pump rate, and getting updates that you're
missing the deadlines and will be slowed from your requested
rate.
dino: We tried running rAF at 120hz, but some animations *assume*
it runs at 60hz and don't actually check the time delta, so
they just run double-speed and break.
dino: We also determined that 120hz animations do use a ton more
power.
dino: And there's some intermediate stuff - if you run at 120hz,
but determine there was no change, don't repaint the screen.
flackr: Maybe speculate that it doesn't change, and throttle down?
dino: We see today's code, where people just run a rAF at all
times...
Rossen: Speculating throttling based on update is tricky -
position sticky needs fast updates, but does nothing until
the element sticks...
Rossen: Switching topics - you mentioned we've been doing this for
2.5 years and we could have been doing more declarative
things.
Rossen: We couldn't have had this discussion two years ago. New
platform we've built up - typed om, worklets... we didn't
know what worklets were two years ago.
dino: I still don't.
Rossen: For our impl it looks like this'll be tricky...
Marshalling state back and forth.
Rossen: We make some decisions about when to offload to compositor
vs main thread based on different pressures, and now we
have to sync the worklets and marshall state, and it's yet
another load of complexity to handle.
Rossen: Not impossible, but certainly tricky.
Rossen: Things scaring me at face value are the sync of worklets
state between threads (or processes).
franremy: We don't yet have Web Animations in Edge. Since that's a
prereq for this, there's some work to do before we can
even think about this.
Rossen: So next state...
flackr: We experimentally implement! And I'd love to see more
detail on the sync issues. There's very little data being
synced right now.
iank: We have the bare bones of an implementation at the moment.
It's a separate thread at the moment. Not much to sync.
flackr: Just a few float values.
iank: We're hoping to have a decent experimental impl by Q4.
[assent that people are generally much happier with this API than
the earlier Tokyo version; issues were called out and asked to
be reported in more detail]
[till: issues about speculation and additional UA state that can
be passed in; rossen: issue about concerns with sync; dino:
issues about timeline pumping rate and warning about missing
deadlines]
<br>
TypedOM
=======
Scribe: fantasai
Should StyleMap be case sensitive?
----------------------------------
GitHub: https://github.com/w3c/css-houdini-drafts/issues/309
TabAtkins: Should the StyleMap be case sensitive for property
names?
TabAtkins: .style isn't
TabAtkins: It doesn't care about casing.
TabAtkins: We do not care which way it goes, it's trivial.
TabAtkins: So asking for opinions.
fantasai: What does it even mean for it to be case-sensitive?
TabAtkins: Only accepts lower case.
dbaron: Is it clear that the canonical case for CSS is lower-case?
...
surma: Interaction with custom properties?
TabAtkins: Custom properties are always case-sensitive.
fantasai: I'll settle this for you: I'll object to treating CSS
properties as case-sensitive, since they are defined
everywhere else as case-insensitive.
dbaron: I think it would be really weird for this to be
case-sensitive.
dbaron: Don't want TypedOM to be different from other OM
dbaron: people sometimes pass properties to functions.
RESOLVED: StyleMap for CSS properties is ASCII case-insensitive
because that's how CSS property names are defined.
Semantics of CSSURLImageValue constructor
-----------------------------------------
Github: https://github.com/w3c/css-houdini-drafts/issues/424
TabAtkins: Things that construct values based on URLs
TabAtkins: Should we do some early parsing of URL and throw an
exception if it fails URL parsing?
TabAtkins: Our opinion is yes.
iank: If you construct a CSSURLImageValue in a PaintWorklet, and
invalid, can't use it. What should semantics be?
TabAtkins: Constructed parsers are used in worklets.
TabAtkins: Impls does something about this in ServiceWorkers
TabAtkins: Shouldn't be difficult to throw if URL is bad.
plinss: This is never synchronous?
TabAtkins: No resolution.
Rossen: Any opinions?
<dbaron> I don't really know the current rules for Gecko's URL
parser, fwiw.
RESOLVED: CSSURLImageValue that fails to parse as a URL should
throw an error.
fantasai: Undecided which error?
plinss: Should have a platform-wide error for URL parsing error.
iank: Platform is very inconsistent, some throw TypeError, others
throw SyntaxError.
dbaron: Sure ServiceWorker relies on this?
[iank cites stuff]
Publication
-----------
fantasai: Were changes from redesign of TypedOM API from Japan
folded in?
TabAtkins: Yes.
fantasai: Has been published?
TabAtkins: No.
fantasai: Publish?
Rossen: After done with topic, sure.
[nainar asks a question, Tab responds with something about
threading, and plinss comments quietly]
TabAtkins: all done
RESOLVED: Publish TypedOM
Implementations
---------------
dino: Status of implementations?
Rossen: Under consideration,
nainar: We held off on implementation until the spec rewrites were
done.
dino: But you have a core behind a flag in the engine?
nainar: Yeah.
nainar: We'll have more at the end of Q4?
dino: Plan to expose as you do with other things?
iank: Depends for TypedOM, maybe not?
iank: We do an origin trial when we think it'll give meaningful
feedback from developers, as opposed to ppl just trying it
out.
iank: If we're getting meaningful feedback from people using it on
Canary and asking what people think about it, then we will
just go into a shipping state directly.
iank: Typically we've found origin trials are useful when we need
ppl to try live with real users.
iank: Animation worklet will go through origin trial
iank: because want to see perf numbers.
dino: But this not so much.
iank: yeah.
iank: Good counter-example would be font variants
iank: will likely go straight to ship.
dbaron: We had an intern working on it last summer, but never
landed.
dbaron: maybe something this summer, but not sure.
jet: We kinda rewrote our style system in a new programming
language that we also wrote, so...
dino: We haven't really stated.
dino: Apple would like to start on this soon, it's one of the
things we're excited about.
Received on Sunday, 27 August 2017 18:32:54 UTC