W3C home > Mailing lists > Public > www-style@w3.org > August 2017

[Houdini] Minutes Paris F2F 2017-08-01 Part I: AnimationWorklet, TypedOM

From: Dael Jackson <daelcss@gmail.com>
Date: Sun, 27 Aug 2017 14:31:58 -0400
Message-ID: <CADhPm3uj4OLiBN7GoyTSOAX+xH_jgMSe=KivMp=UEEa9-iJDKw@mail.gmail.com>
To: public-houdini@w3.org
Cc: www-style@w3.org
   These are the official Houdini Task Force
     minutes. Unless you're correcting the
      minutes, please respond by starting
 a new thread with an appropriate subject line.


  - flackr introduced the newest iteration on the spec which takes
      into account previous feedback that it wasn't fitting well
      into web animations.  Spec here: https://wicg.github.io/animation-worklet/
  - General response was very positive that this seemed like it
      would solve use cases well and integrate with existing specs.
  - There was conversation around exactly what data on user state
      should be passed in; till will create a github issues to
      continue this conversation.
  - Detecting when a user is in a mutating state is another topic
      that will be added to github for conversation by Rossen.
  - Lastly, the spec will need to detail how to address when how to
      handle when a specific timeline is requested, but the goal
      isn't being met so timeline is being scaled down. Dino will
      file an issue on this topic.


  - RESOLVED: StyleMap for CSS properties is ASCII case-insensitive
              because that's how CSS property names are defined.
              (Issue #309)
  - RESOLVED: CSSURLImageValue that fails to parse as a URL should
              throw an error. (Issue #424)
  - RESOLVED: Publish TypedOM
              -> https://www.w3.org/TR/2017/WD-css-typed-om-1-20170801/


Agenda: https://github.com/w3c/css-houdini-drafts/wiki/Paris-F2F-August-1st-2017

  Rossen Atanassov, Microsoft
  Tab Atkins, Google
  David Baron, Mozilla
  Brian Birtles, Mozilla
  Tantek Çelik, Mozilla
  Emil A Eklund, Google
  Elika Etemad, Invited Expert
  Rob Flack, Google
  Koji Ishii, Google
  Dean Jackson, Apple
  Ian Kilpatrick, Google
  Peter Linss, Invited Expert/TAG
  Jack Moffitt, Mozilla
  Naina Raisinghani, Google
  François REMY, Microsoft
  Simon Sapin, Mozilla
  Nicolas Silva, Mozilla
  Alan Stearns, Adobe
  Surma, Google
  Sergio Villar, Igalia
  Jet Villegas, Mozilla
  Eric Willigers, Google

Scribe: TabAtkins

  <jack> https://wicg.github.io/animation-worklet/
  flackr: A while back we presented a CSS version of AnimationWorklet.
  flackr: Since then we've taken the concerns about this fitting
          with Web Animations and turned it into a more imperative
          API, like a custom Web Animation.
  flackr: The construction takes the name of an animator, like the
          PaintWorklet api, and an effect in the timeline.
  flackr: Then the animator can do whatever it wants to manipulate
          the effect.
  flackr: This gives the same flexibility as Web Animation, letting
          you manipulate multiple elements as a group effect.
  flackr: This brings more benefits, too.
  flackr: No longer have to worry about the final output value, this
          is just part of the effects stack.
  flackr: Now it can run between other main-thread animation
          effects, run out of sync, etc.

  <flackr> https://googlechrome.github.io/houdini-samples/animation-worklet/parallax-scrolling/?nojank
  flackr: This parallax demo shows simplest example.
  flackr: Construct an effect that can affect transformY and
          transformX, read the scroll offset, and update the
  flackr: This can be done with just scrollTimeline and a regular
          web animation, but with this you can do more complex
          effects on top as well.
  flackr: Because we're using a single input timeline, you can
          explain how the animation calls finish/reverse/playback
          rate affects the animations.
  flackr: We allow in the spec passing extra timelines, and
          attaching/detaching them.
  flackr: Like an animation that invalidates on scroll, which then
          attaches itself to the document timeline for a little
          while before removing itself. A scroll-trigger animation,
          rather than scroll-driven.

  <flackr> https://googlechrome.github.io/houdini-samples/animation-worklet/spring-sticky/
  flackr: This example invalidates on scroll, then attaches itself
          to the document timeline to continue until it finishes,
          then detaches.
  flackr: Another benefit of this is that it's pretty easy to
          polyfill. Doing the CSS version was very gross, had to
          walk the tree.
  flackr: Other than running on the main thread, the polyfill is
          quite performant now.

  iank: Nice thing about this model now is that there's conceptually
        only one timeline we're listening to at a given moment, the
        one the animation gets invalidated on.
  iank: Could be scroll, could be document.
  <iank> https://googlechrome.github.io/houdini-samples/animation-worklet/spring-sticky/spring-sticky-animator.js

  [discussion of demo]
  iank: So looking at demo, new WorkletAnimation gives the name of
        the animator
  iank: And an array of effects.
  iank: Behind the scenes, we bump it up into a group effect.
  iank: You get passed one timeline, the primary timeline; in this
        case it's scroll.
  iank: And you can pass in an options bag, it's structured cloned
        over to the animation worklet.
  iank: And inside the springStickyAnimator.js, there's a
        registerAniamtor call.
  iank: Its constructor takes the object bag, pulls out the
        documentTimeline from it.
  iank: It animates on the scroll timeline.
  iank: Then it attaches and detaches from the documentTimeline to
        make it invalidate properly (and finish up).
  iank: To actually apply effect you set the localTime of each
        individual effect that's passed in.
  iank: This integrates nicely if we ever have sequential group
        effects in web animations, you'll just set the localTime of
  iank: Because we're integrating the web animation timelines, play/
        pause/etc just falls out.

  iank: This example also has some internal state in the class.
  iank: There's a velocity and position array.
  iank: So in this case the springs can keep track of velocity
        across invocations.
  <astearns> I kind of understand the scroll down animation in the
             second example. The little jump when you scroll up is
             pretty weird, though.
  <surma> astearns: That is most likely the polyfill in action. IIUC
          a native implementation wouldn’t have that jitter

  dino: When the animation detaches itself from document timeline,
        that doesn't cause the animation worklet to go away, right?
  flackr: Right, it's still attached to the scroll timeline.
  dino: And what makes it attach again?
  flackr: It's always attached to the scroll timeline.
  flackr: In the HTML file you specify the scroll timeline as the
          primary; you just attach to the document timeline manually
          when the animation starts.
  dino: What's the currentTime that's passed in?
  iank: The main timeline, scroll in this case.
  dino: So attaching to the document timeline just says, call me on
        every document tick.
  iank: Yeah.
  iank: There's another way to do this - you can attach to the
        documentTimeline as primary, then just read the
        scrollTimeline on every tick. That's not as performant
  iank: This way, you can declare when you care about continuous
  iank: We could avoid the performance issues of passing in the
        document timeline by waiting on animation triggers and
        self-destroying, but that's more complicated.
  dino: And the way the animations move is you set localTime on the
        effect, which affects scroll position...
  iank: Yes.

  dino: I like that this is all done on web animations.
  dino: We do our scrolling in a separate process.
  dino: If possible we don't want to send client JS to that process;
        exposes us to exploits.
  dino: But the animation code still needs to run somewhere...
  flackr: In Chrome I think we'll have a thread for the animation
          worklets, and send it work - these timelines have new
          values, run and get back the local times produced.
  flackr: Don't think we'll run it in our scroll-handling thread.
  iank: We also have deadlines, so slow script'll just not work.
  iank: And we'll fall back to the main thread if animating a
        non-accelerating property.
  dino: Is sticky okay if running in a non-scrolling thread?
  flackr: We haven't found it to be an issue. It seems to be
          possible to pass back and forth state before running out
          of time.
  iank: We have a deadline of a few ms to let things finish; as long
        as it hits that deadline we can keep it in sync.

  dino: This all looks great; it kinda concerns me that - this is
        more of a metacomment - we've spent 2.5 years speccing
        low-level stuff without getting much, when we could have
        done high-level stuff by now.
  dino: Lots of code here to do parallax, could have been done very
        simply previously.
  dino: Worries me a bit how much complexity there is to do
        relatively simple things. This sticky spring isn't something
        we'd want to build in, obviously, but parallax could have
        been easy.
  birtles: I don't think that's quite the case. There's a piece
           between, Scroll-Driven Animations, and parallax could
           just build on that. It has both an API and a declarative
  dino: I'm saying we could have done less work just doing parallax.
  birtles: I think this is trying to join dots - if the built-in
           doesn't cover your use-case, can you extend it out.
  birtles: I think this helps validate the scroll animation proposal.
  iank: I think previous attempts, too, weren't as simple. Didn't
        have an easy way to listen to scrolls, for example.
  dino: My only concern is that I wish we could produce some logic I
        could send to the scroll thread without having to send JS...
  TabAtkins: I'm really happy with how general this is, because it
             means we can delegate a lot of hard cases to it, and
             then do Scroll-Linked Animations limited to the simpler
             cases. It was way too difficult ticking all the boxes
             with just SLA when I tried it in the past; there's a
             nice simple core we can limit ourselves to now.
  dino: That sounds good.

  birtles: There are some details here I'm not sure about, like
           blessing one timeline above others, but these are details
           we can work on. I think the approach is good overall, and
           I'm happy it's working on top of web animations.
  birtles: I didn't realize the spec had been substantially updated,
           so I'm still going over those issues.
  dino: What do you think about implementability?
  birtles: Dunno yet.

  till: The internal state that has to be transferred to a worklet
        means that it's not possible to ever parallelize this
  till: If instead we did something like provide a velocity of the
        current scroll, or last location, we could potentially do
        away with internal state, and speculatively execute multiple
        steps in the timeline, and use them if necessary.
  flackr: It's worth pointing out that you can run each animator
  till: Understood.
  flackr: We'd have to specify all different pieces of state ...
  TabAtkins: Not necessarily all, just some useful stuff. With a
             properly designed API, could detect when they're only
             relying on pure state.
  till: Or restrict them to only pure state, if that's not too
  flackr: Another header example is where your action depends on
          scroll direction in the past...
  till: Right, we could pass that in.
  TabAtkins: Or just, instead of mutating, always have it run on
             pure state that it can produce new versions of.
  flackr: Yeah, maybe your animation could specify that it's clean,
          and we could have a policy of regularly disposing of
          "clean" animations.
  till: This isn't about tearing down an animation, it's about
        running multiple frames interleaved, and reducing the risk
        of getting jank in this one animation. That's only possible
        with pure state.
  TabAtkins: Yeah, this could easily be an array of pure position/
             velocity state, and just returning an updated clone.
  iank: A concern was the cost of structured cloning every frame.
        Would be better if we could just clone every N frames.
  flackr: This is coming from the world where devs would usually
          write this as a main-thread effect, so here at least they
          can be run separately.
  till: Right, so I see that being able to keep internal state is
        more powerful, but it keeps you from parallelizing.
  flackr: I wonder if there's verification possibility - analyze the
          script to see if it's pure enough.
  till: No, we tried that with parallel.js
  TabAtkins: And it's easy to fall out of the pit of success for no
             apparent reason.
  iank: So if we returned the state every frame from animate, and
        passed it back in, we could do pure state.
  till: Not quite - we want to ask for frame N, *and* start working
        on frame N+1, without waiting for frame N to finish.
  till: You could increase your frame budget from 16ms to
        32ms if you're doing frames on 2 different cores
  TabAtkins: Ah, so that does require only relying on UA-provided
  till: Yeah, letting us take care of animations on multiple cores.

  surma: People are already asking for how to depend on user input.
         That breaks speculation, right?
  till: People don't usually change the velocity of their input in
        short timescales (30ms or so). You'll occasionally
        mis-speculate, but mostly will be okay.
  birtles: And doing multiple frames at slight offsets to produce
           motion blur.
  TabAtkins: I think this is all quite tractable with minimal

  iank: Would you be okay with starting with a mode that says you
        only use pure state, and we'll tear down your object
        regularly to prevent you from relying on impure state?
  till: I probably can't convince you to actually produce fresh
        objects on every frame, so yeah.
  TabAtkins: Get all the JS engines to give us cheap immutable
             state, and we'll talk. ^_^
  till: And spec some additional pure state info the UA can provide
        to the animator.
  iank: Velocity, acceleration, maybe last N frames of scroll data...
  iank: So thinking maybe we start with a separate mode, see what
        people are using with it, start adding more data.

  jack: Could we pass in velocity and acceleration by default? You
        pretty much have to calculate that anyway.
  till: Problem with the flag is that we can't default to that.
        Maybe if we just speculate wrongly for the first N frames we
        can switch to the stateful version?
  flackr: We have this idea that we can add info to the timeline
          that contains this extra info, like user input too.
  iank: We could invert the flag - default to not passing in new
        state, you can flag it on to get user-defined state passed
  till: Seems useful regardless of speculative execution.
  till: If we always pass in these values, people can rely on it,
        and we can just assume things are pure unless we fail
  <iank> https://gist.github.com/anonymous/e22250dbdfa92da59508a220b8087d9d

  TabAtkins: How do you speculate if people *are* using mutating
             state? Seems you'd mess up the state.
  till: Run one thread as source of truth for a while, speculatively
        run the same frames ahead of time on another thread. If
        speculation fails for more than N out of the first M frames (
        speculation gives different result than the "real" thread),
        assume they're using mutating state and stop speculating.
        Otherwise, continue speculating.
  till: Something that could be helped by sequential speculative
        execution is power usage - better to spike the CPU in bursts
        rather than run it continuously.
  dino: And with 120Hz animation, letting the animator idle as much
        as possible would be good.
  till: And with 4 cores, you can split frames between them, have
        better guarantee of hitting every frame.
  dino: So I think you're saying - based on what's happening now,
        you'll be issuing a lot of updates in a short amount of
        time, so you want to use other cores to pregen some frames
        ahead now, so they're ready when needed.
  dino: So the speculative ones return frames, timestamped, the
        system uses them?
  TabAtkins: Yeah. Speculate for the first N frames along with
             primary thread, compare values; assume speculation is
             safe if it succeeds.
  dino: Why not let the worklet return multiple successive frames?
  TabAtkins: That doesn't solve the problem, right? You're still
             spending the same amount of CPU as calling it multiple
  dino: Like, if I know the animation wants to run at 120Hz, but
        I'll be called at 60Hz, you can do two frames at once.
  till: That still requires you to calculate frames at 120Hz. With
        speculation you can actually pump frames at 60Hz, just every
        other frame on each core.

  TabAtkins: Probably useful to have a flag that specifies whether
             you think you're safe to speculate or not. If you claim
             you are, we do some things to keep you honest (tear
             down your object regularly); if you claim you're not,
             UAs might still do speculation speculatively to see if
             you really are, as till described earlier.
  flackr: This sounds sensible.
  till: This is also based on impl experience we're having with
        Paint Worklet, we're already doing speculation there.
        Slightly different cases there, but similar API.
  <till> PR for speculative paint worklet execution:
  dino: Also would like a flag specifying how fast timeline should
        pump. rAF is kinda 60Hz, but we now have variable-rate
        screens coming out, etc.

  dino: So question is how to specify that from a worklet - message
        to the timeline saying "I wanna get called back more
        frequently/in at least 4ms"? And some feedback saying you're
        not meeting that goal, so you're getting scaled back.
  flackr: Would you ever not want to run that fast?
  dino: Yes, if your animation doesn't need to be 120hz, it's better
        to run at 60hz for power reasons.
  <brucel> +1 to what dino said "if your animation doesn't need to
           be 120hz, it's better to run at 60hz for power reasons."
           - Opera desktop, for example, has power saving mode in
           which frame rate for video and animations is deliberately
  TabAtkins: This kinda sounds like various stream APIs, which let
             you attach to timelines at various coarseness.
  dino: and figuring out how fast a timeline *can* be pumped, asking
        for a particular pump rate, and getting updates that you're
        missing the deadlines and will be slowed from your requested
  dino: We tried running rAF at 120hz, but some animations *assume*
        it runs at 60hz and don't actually check the time delta, so
        they just run double-speed and break.
  dino: We also determined that 120hz animations do use a ton more
  dino: And there's some intermediate stuff - if you run at 120hz,
        but determine there was no change, don't repaint the screen.
  flackr: Maybe speculate that it doesn't change, and throttle down?
  dino: We see today's code, where people just run a rAF at all
  Rossen: Speculating throttling based on update is tricky -
          position sticky needs fast updates, but does nothing until
          the element sticks...

  Rossen: Switching topics - you mentioned we've been doing this for
          2.5 years and we could have been doing more declarative
  Rossen: We couldn't have had this discussion two years ago. New
          platform we've built up - typed om, worklets... we didn't
          know what worklets were two years ago.
  dino: I still don't.
  Rossen: For our impl it looks like this'll be tricky...
          Marshalling state back and forth.
  Rossen: We make some decisions about when to offload to compositor
          vs main thread based on different pressures, and now we
          have to sync the worklets and marshall state, and it's yet
          another load of complexity to handle.
  Rossen: Not impossible, but certainly tricky.
  Rossen: Things scaring me at face value are the sync of worklets
          state between threads (or processes).
  franremy: We don't yet have Web Animations in Edge. Since that's a
            prereq for this, there's some work to do before we can
            even think about this.
  Rossen: So next state...
  flackr: We experimentally implement! And I'd love to see more
          detail on the sync issues. There's very little data being
          synced right now.
  iank: We have the bare bones of an implementation at the moment.
        It's a separate thread at the moment. Not much to sync.
  flackr: Just a few float values.
  iank: We're hoping to have a decent experimental impl by Q4.

  [assent that people are generally much happier with this API than
      the earlier Tokyo version; issues were called out and asked to
      be reported in more detail]
  [till: issues about speculation and additional UA state that can
      be passed in; rossen: issue about concerns with sync; dino:
      issues about timeline pumping rate and warning about missing


Scribe: fantasai

Should StyleMap be case sensitive?
  GitHub: https://github.com/w3c/css-houdini-drafts/issues/309

  TabAtkins: Should the StyleMap be case sensitive for property
  TabAtkins: .style isn't
  TabAtkins: It doesn't care about casing.
  TabAtkins: We do not care which way it goes, it's trivial.
  TabAtkins: So asking for opinions.
  fantasai: What does it even mean for it to be case-sensitive?
  TabAtkins: Only accepts lower case.
  dbaron: Is it clear that the canonical case for CSS is lower-case?
  surma: Interaction with custom properties?
  TabAtkins: Custom properties are always case-sensitive.
  fantasai: I'll settle this for you: I'll object to treating CSS
            properties as case-sensitive, since they are defined
            everywhere else as case-insensitive.
  dbaron: I think it would be really weird for this to be
  dbaron: Don't want TypedOM to be different from other OM
  dbaron: people sometimes pass properties to functions.

  RESOLVED: StyleMap for CSS properties is ASCII case-insensitive
            because that's how CSS property names are defined.

Semantics of CSSURLImageValue constructor
  Github: https://github.com/w3c/css-houdini-drafts/issues/424

  TabAtkins: Things that construct values based on URLs
  TabAtkins: Should we do some early parsing of URL and throw an
             exception if it fails URL parsing?
  TabAtkins: Our opinion is yes.
  iank: If you construct a CSSURLImageValue in a PaintWorklet, and
        invalid, can't use it. What should semantics be?
  TabAtkins: Constructed parsers are used in worklets.
  TabAtkins: Impls does something about this in ServiceWorkers
  TabAtkins: Shouldn't be difficult to throw if URL is bad.
  plinss: This is never synchronous?
  TabAtkins: No resolution.
  Rossen: Any opinions?
  <dbaron> I don't really know the current rules for Gecko's URL
           parser, fwiw.

  RESOLVED: CSSURLImageValue that fails to parse as a URL should
            throw an error.

  fantasai: Undecided which error?
  plinss: Should have a platform-wide error for URL parsing error.
  iank: Platform is very inconsistent, some throw TypeError, others
        throw SyntaxError.
  dbaron: Sure ServiceWorker relies on this?
  [iank cites stuff]


  fantasai: Were changes from redesign of TypedOM API from Japan
            folded in?
  TabAtkins: Yes.
  fantasai: Has been published?
  TabAtkins: No.
  fantasai: Publish?
  Rossen: After done with topic, sure.

  [nainar asks a question, Tab responds with something about
      threading, and plinss comments quietly]
  TabAtkins: all done

  RESOLVED: Publish TypedOM


  dino: Status of implementations?
  Rossen: Under consideration,
  nainar: We held off on implementation until the spec rewrites were
  dino: But you have a core behind a flag in the engine?
  nainar: Yeah.
  nainar: We'll have more at the end of Q4?
  dino: Plan to expose as you do with other things?
  iank: Depends for TypedOM, maybe not?

  iank: We do an origin trial when we think it'll give meaningful
        feedback from developers, as opposed to ppl just trying it
  iank: If we're getting meaningful feedback from people using it on
        Canary and asking what people think about it, then we will
        just go into a shipping state directly.
  iank: Typically we've found origin trials are useful when we need
        ppl to try live with real users.
  iank: Animation worklet will go through origin trial
  iank: because want to see perf numbers.
  dino: But this not so much.
  iank: yeah.
  iank: Good counter-example would be font variants
  iank: will likely go straight to ship.

  dbaron: We had an intern working on it last summer, but never
  dbaron: maybe something this summer, but not sure.
  jet: We kinda rewrote our style system in a new programming
       language that we also wrote, so...

  dino: We haven't really stated.
  dino: Apple would like to start on this soon, it's one of the
        things we're excited about.
Received on Sunday, 27 August 2017 18:32:54 UTC

This archive was generated by hypermail 2.4.0 : Friday, 25 March 2022 10:09:08 UTC