W3C home > Mailing lists > Public > www-style@w3.org > May 2017

[Houdini] Minutes Tokyo F2F 2017-04-18 Part I: Worklets, Animation Worklet

From: Dael Jackson <daelcss@gmail.com>
Date: Mon, 15 May 2017 04:40:58 -0400
Message-ID: <CADhPm3vQAeMDmtnuJuPio3U5Sr_N02oe2tCB86Quhw5p+JxAPw@mail.gmail.com>
To: public-houdini@w3.org
Cc: www-style@w3.org
=================================================
   These are the official Houdini Task Force
     minutes. Unless you're correcting the
      minutes, please respond by starting
 a new thread with an appropriate subject line.
=================================================


Worklets
--------

  - RESOLVED: import() to be renamed to addModule().
  - RESOLVED: Change preferred import to use <link> over <script>.
  - RESOLVED: Publish a new working draft for worklets, pending the
              changes we discussed today.

Animation Worklet
-----------------

  - Some potential uses of an animation worklet were shown:
      http://googlechrome.github.io/houdini-samples/
  - There was debate as to if this should live on its own or be a
      part of web animations which will need more discussion.

===== FULL MINUTES BELOW ======

Agenda: https://github.com/w3c/css-houdini-drafts/wiki/Tokyo-F2F-April-18-2017

Present:
  Rossen Atanassov, Microsoft
  Tab Atkins, Google
  L. David Baron, Mozilla
  Brian Birtles, Mozilla
  Bogdan Brinza, Microsoft
  Rick Byers, Google
  Emil A Eklund, Google
  Elika Etemad, Invited Expert
  Rob Flack, Google
  Simon Fraser, Apple
  Dean Jackson, Apple
  Ian Kilpatrick, Google
  Chris Lilley, W3C
  Peter Linss, Invited Expert
  Jack Moffitt, Mozilla
  Xidorn Quan, Mozilla
  Hiroshi Sakakibara, BPS
  Simon Sapin, Mozilla
  Alan Stearns, Adobe
  Shane Stephens, Google
  Majid Valipour, Google
  Lea Verou, Invited Expert
  Jet Villegas, Mozilla
  Philip Walton, Google
  Masataka Yakura, observer

  Scribe: eae

Worklets
========

  Rossen: Let's start with worklets.

import() name is potentially confusing
--------------------------------------

  <iank> https://github.com/w3c/css-houdini-drafts/issues/374
  iank: There is the feedback from various people that the import
        method on worklets is confusing, similar to import statement
        in javascript which is somewhat similar but not really. Any
        preferences for another name? Looking for feedback.
  iank: Leading proposal is Worklet.spawn
  Rossen: What is the history here? Feedback from people trying to
          use it?
  iank: Mostly feedback from annevk.
  iank: The main push back was that the script can end up running
        multiple global scopes which is a bit unique.
  iank: This was seen as confusing.
  philipwalton: Definitively not import.

  iank: Any other preferences?
  philipwalton: Could you clarify what it would be spawning if we go
                with spawn.
  iank: It is not really creating anything new, it loads a script
        into multiple global scopes. Not really spawning.
  iank: Another suggestion is loadModule. Primary push back there is
        that it looks similar to system.import which loads into the
        current global scope.
  philipwalton: Any discussion around how this would interact with
                ES6 modules?
  iank: Would just work.
  philipwalton: Other workers have the importScript function?
  iank: Yes and that would load into the current global scope.
  iank: dbaron, any preferences?
  dbaron: Not really.

  philipwalton: If the only difference is that it loads into
                multiple global scopes the name should reflect that.
  dbaron: When you described what it does you used the word load....
  iank: Propose that we switch it to loadModule and see if there is
        any pushback for that specific name.
  Rossen: Sounds good to me.
  jack: I don't like spawn.
  flackr: Sounds like ES6 load module but does something
          differently, my only concern.
  iank: You need to have at least one global running. Your code will
        get immediately executed but if another global scope is
        spawn it'll load into that as well.
  iank: The paint specification for instance requires at least two
        global scopes at all times (unless under memory pressure).

  rbyers: Perhaps register instead?
  flackr: addModule?
  iank: I could live with addModule.
  iank: Will update github issue to propose addModule and await
        feedback.
  Rossen: addModule is the final proposal for now?
  philipwalton: I kind of like registerModule better but add is fine.
  Rossen: In summary, addModule is much preferred over import or
          spawn. Unless there are other proposal I suggest we
          resolve to rename it to addModule. Any objections?

  RESOLVED: import() to be renamed to addModule()

<script> tag import feature design
----------------------------------

  <iank> https://github.com/w3c/css-houdini-drafts/issues/376
  iank: This topic is the design of the import. We could use a
        script tag or a link tag.
  iank: The script tag basically works as outlined in top comment of
        issue. Effectively you'd have a new type, i.e.
        type=paintworklet, always loads as module. Either with a src
        or inline.
  iank: The link tag version is basically what service worker does.
        Dominic outlined the proposal in comment 3 on the issue.
  iank: There are pros and cons for each.
  iank: Pro for script tag is that it matches what people do today
        and works with data urls. The downside is that it is running
        some code in a different global scope which is new and
        changes the script process model.
  iank: It is easier for implementers to implement the link type but
        more powerful for authors to use a script tag.
  eae: Optimizing for what is best for web developer has my vote.
  Rossen: No opinions?
  <nainar> Same for me
  iank: I could also defer to TAG on this one.

  dbaron: My only thought is that if there is a similarity with
          service worker it would be nice to use the same
          functionality with different names rather than have
          different approaches.
  till: Related, the hack with the two instances of a worklet to
        avoid global state. Clearly not ideal but better than not
        having a way to discourage state. A way to maybe deal with
        this would be to have an execution context that cannot store
        global state. The syntax for loading worklets doesn't allow
        us to change how they are loaded right now. Would it make
        sense for this to be versioned to possibly allow for another
        execution context down the line?
  iank: I think both proposals could handle that.

  iank: Taking dbaron's feedback into account it sounds like the
        link tag is preferable.
  dbaron: I don't know how close this is to service worker.
  dino: Closer to service worker than inline so makes sense.
  philipwalton: You said in the first comment that there was various
                sources of feedback. Was any of it about supporting
                situations with js enabled? Would a link tag work in
                those cases?
  iank: Think that is a UA decision.
  iank: If browsers have a preload scanner that could apply to
        <link> tags.
  iank: With a <script> tag and data url the memory cost would be
        higher, possibly.
  philipwalton: link makes more sense to me.
  Rossen: Sounds like we have consensus around link.

  RESOLVED: Change preferred import to use <link> over <script>

Adding CSP to worklets
----------------------

  iank: Adding CSP to worklets.
  till: One further question. What is the plan for when to use the
        different global scopes.
  iank: Initially for paint we'll likely have two globals and might
        shut one of them off and re-spawn occasionally.
  iank: When we see issues we plan to limit it to one under memory
        pressure, the exact policy is not yet clear.
  jack: One thing that would be nice would be to make the spec
        language allow other approaches than using two global
        scopes. I.e. if there is another way to limit global state.
  iank: My opinion is that frozen realms would be hard to implement.
        Freezing globals on the global scope on start might work but
        it has some down sides.
  dbaron: Would frozen realms cause JS errors where there weren't
          any before?
  till: Yes.

  iank: The one piece missing is the requirement that all of the
        globals share one fetch response map.
  iank: When I did the first pass of worklets it was reasonable easy
        to separate.
  dino: Has anyone else started implementing?
  iank: I thought there was some momentum on the firefox side for
        paint worklets.
  Jet: We have an implementation going.
  <jet> Mozilla Worklets bug:
https://bugzilla.mozilla.org/show_bug.cgi?id=1329166
  dbaron: There was some implementation work for worklets themselves
          but nothing specific yet.

  Rossen: Is there anything on the status updates you want to fill
          in?
  iank: No.
  Rossen: The current working draft is almost a year old, could you
          update that please?

  RESOLVED: Publish a new working draft for worklets, pending the
            changes we discussed today

Animation Worklet
=================

  <nainar> The spec is here: https://wicg.github.io/animation-worklet/
  [flackr to present a demo using dbaron as a proxy]
  <flackr> polyfilled demos are at:
http://googlechrome.github.io/houdini-samples/
  <rbyers> https://lists.w3.org/Archives/Public/www-style/2014Feb/0769.html

  rbyers: Brief introduction. First of all, a little over three
          years ago I said we didn't want to do snap points in blink
          and instead focus on primitives. I was wrong. Took much
          too long for us to get the primitives. We have now shipped
          position sticky and snap points the same way as everyone
          else. We still want to do the primitives though so that we
          don't have to do one offs for all future features like
          this. We want the low level primitives to be available, a
          way to change transforms based on custom properties and
          scroll position off the main thread.
  rbyers: Opacity as well and there could be more.
  flackr: The idea is to present what is currently possible using
          low level primitives.
  rbyers: At TPAC we presented a slimmed down version, no longer
          runs on the scroll thread instead uses a worklets. Do not
          want to special case and do what animations do. Allow UA
          to choose to run off main thread or not depending on
          capabilities.
  rbyers: Focus on performance isolation for animations from JS.
  rbyers: We have a polyfill and demo and hope to have an origin
          trial and data to share by the time we meet again in Paris.
  rbyers: Big picture, any questions before we do the demo?

  rbyers: Origin trials is a mechanism in chrome to allow developers
          to experiment with experimental APIs for a limit time and
          with limited reach. Was an idea at TPAC a few years ago.
          Now in chrome.
  rbyers: Time limited and usage limited, can be taken away at any
          point.
  dbaron: The one other comment, it would be useful to have the big
          picture summary in the spec. Rather than going straight
          into details.
  rbyers: In the spec and not the explainer?
  dbaron: The part about which thread it runs on in particular.
  dbaron: Deserved to be in an introductory section. The explainer
          also has not been updated.
  rbyers: Will file issue about it.

  flackr: If you go into the examples and open up the spring one and
          then bring up the dev tools to show the source.
  flackr: http://googlechrome.github.io/houdini-samples/animation-worklet/spring-timing/
  flackr: This specifies the name of an animator which will be
          accelerated.
  flackr: Then you can have custom properties that are sent over.
  flackr: Call to import the worklet, polyfill-animation-worklet
  flackr: We register the animator with the name we specify in the
          other file. Name of the slots. Outputs transforms. This
          declares which timelines feed into the worklets. Plan to
          support two timelines, document (performance.now) and
          scrolling.
  flackr: You can specify the timeline.
  flackr: CSSOM transforms are used to avoid string parsing.
  flackr: The idea is that when you instance this animation every
          time you want to produce a new animation frame you would
          call this animate function with the timeline. It will
          change some of the properties, if they are all fast
          properties it would go on the compositor thread, if not
          it'll go on the main thread. It can have a deadline, if
          the animate calls are taking too long we can chose to use
          the previous values which would update in a performance
          isolated fashion but no longer be in sync. Shows the basic
          idea.
  flackr: Some of the other demos shows how it would work with
          scroll input.

  smfr: I would prefer to see this use web animations.
  smfr: I don't think we want a whole separate way to do animations.
  rbyers: We don't want to limit us to the expressiveness of web
          animations.
  smfr: Then shouldn't we expand web animations to support that?
  iank: Could plug into web animations.
  iank: You could pass in all the element and that would be your
        hook into the web animation.
  flackr: I should have mentioned that, we have a plan for a way to
          express this as a web animation.
  majidvp: Another idea is that perhaps we can instantiate a
           keyframe effect and assign it as output and use a
           different effect to sample and output. We could have a
           state here where you.
  flackr: We have examples that show where you do more complex
          effects.

  birtles: Seems like this example doesn't need state; could just be
           a script timing function.
  birtles: We spoke briefly last night and I'd like to see how the
           pieces align, I don't think this is a good example using
           animation worklets. What if it has state?
  flackr: You can instance separate ones for each root and run them
          in parallel.
  flackr: The idea for state is to store it on the instance of the
          animator, we could have a function that allows you to
          store state before the instance is destroyed.
  birtles: It does restrict a little bit the optimizations that user
           agents can do. I want to make sure that restriction is
           warranted.
  rbyers: That is true for all animations today. No one is saying we
          should eliminate rAF as it limits UA optimizations.

  birtles: What are the uses cases with state that couldn't be done
           without it.
  rbyers: You could probably do most with the combination of snap
          points and scroll linked animations.
  rbyers: Killer use case which we struggled with for years is pull
          to refresh.
  rbyers: Another example is something like google maps where you
          have pinch zoom and panning with tiles. Can't be solved
          with the MVP but we can add onto it.
  rbyers: Original proposal supported this but was too complicated.
  dino: How does maps work today?
  rbyers: Needs to handle input on main thread and struggle with 60
          fps.
  rbyers: With animation worklets panning and scaling could happen
          at 60fps on an animation thread.
  dino: The events are still coming in on the main thread, how would
        that work?
  rbyers: Wouldn't work in the MVP, we'd need to be able to pass
          input events to the animation worklet.
  dino: Seems like a lot of information, not just raw events.
  dino: It is always the only example we come up with. Are there
        other examples that require state?
  rbyers: We have more examples.

  flackr: Sticky spring example is stateful.
  <eae> http://googlechrome.github.io/houdini-samples/animation-worklet/spring-sticky/
  flackr: A couple of boxes that are trying to stick to the viewport
          and each other.
  flackr: Has a rubber band effect, sticking to box above it with
          some springyness.
  flackr: Example of effect where we'd need state.
  flackr: Scroll input is coming to this through the scroll timeline
          that I mentioned earlier.
  flackr: It looks at the nearest scrollable element to the root of
          the animator.
  flackr: It provides this as a scrolling timeline to the animator
          js.
  flackr: Scroll timelines do not tell you the actual scroll offset,
          provided as custom properties.
  flackr: We intentionally been vague about scroll timelines in the
          spec.
  birtles: What's the synchronization concept?
  flackr: You'd get the new values at the next frame.
  flackr: On the main thread you'd have the values from the last
          compositor frame. The result of the previous main frame.
  birtles: The compositor could keep ticking ahead but the main
           thread needs to be kept in lock step.
  flackr: You get the value that was provided by the previous main
          frame.
  flackr: The key thing in this example is that we added state to
          the instance. Velocity and position. Track the momentum
          for each box.
  iank: It should be noted that we could still kill these things but
        we wouldn't do that during a user gesture.

  flackr: We have an example of a case where we need to coordinate
          animations between multiple elements too
  <eae> http://googlechrome.github.io/houdini-samples/animation-worklet/expando/
  flackr: There is a top level element that is a circular clip. The
          second element has a counter effect to the parent that
          clips it. It cannot be exposed in the current system as it
          is the inverse to the other element. This is an example of
          a real effect used by many native applicators. Shows how
          one would coordinate the effect between multiple
          animations.
  flackr: The counter scale is the inverse of the scale of the clip.
  flackr: Clip paths can be complex, can have paths. This one is
          simple, just a circle.
  birtles: I think this could be expressed as a group effect from
           Web Animations level 2 -- assuming that happens.
  flackr: We did this without animation worklets by using a lot of
          keyframes to approximate the inverse effect.
  flackr: Not a linear combination. A curve that you cannot describe
          with the current animation spec.
  flackr: I think this describes that we're thinking.
  majidvp: The slot on the root element is optional.
  majidvp: In this case, the root has some sort of effect.
  flackr: If this was a typical web animation each element would be
          an animation root.
  flackr: To allow for coordination we have roots and non-roots.
  jack: It seems like these examples are the worst possible ways to
        do this. Would be nice if the default did the right thing.
  <dbaron> jack, was the point that you want the animation root as
           deep as possible in the tree?
  <shane> dbaron: I think jack wanted that, yes. Specifically the
          animation root should *be* the animated element unless
          otherwise specified by the author.
  <jack> dbaron: yes. we want default to be maximal parallelism
  <jack> whereas now it is minimal

  smfr: It is far to say that it is using the timeline, should the
        timeline express a hierarchy of some kind?
  smfr: It seems like you are building a slightly declarative
        version of web animations.
  smfr: Not quite explicit relationship between what you are
        describing and web animations.
  flackr: Perhaps we need a way to describe this in javascript as a
          web animation?
  smfr: I'd like to see this built on top of web animations.
  flackr: We have ideas for that, I could write it up.
  rbyers: If we layer it on top of web animations we limit it to web
          animations.
  smfr: Then we should update web animations to support that.
  jack: Specifically the animation root should *be* the animated
        element unless otherwise specified by the author.
  smfr: It still feels to be in general that web animations should
        be the canonical animation framework and the layer things on
        top of it.

  <rbyers> Jack's issue with the bad default for roots:
           https://github.com/WICG/animation-worklet/issues/51
  <rbyers> dbaron's earlier issue with spec introduction:
           https://github.com/WICG/animation-worklet/issues/50
  * astearns thanks rbyers for the real-time issue generation

  rbyers: There was talk awhile ago about custom animations, is that
          dead?
  birtles: At the moment, yes
  rbyers: If we could come up with a way to extend web animations we
          could use that. Would have to be worklet based.
  iank: In my mind the way that this links to web animations is that
        the default web animation coordinator is in the document and
        this can replace that. Instead of multiple timelines you
        have a single timeline for the document.
  birtles: Another direction would be to extend web animations to
           allow script to define timelines.
  flackr: I think of this a script defined timeline.
  birtles: hierarchical effects in this particular case.
  birtles: The ability to target multiple elements.
  iank: What do you see as the capabilities of scripted effects for
        web animations?
  birtles: We had a proposal at some point - you would get timing
           input and could do anything. Was problematic.
  iank: Problematic in what way?
  iank: Assuming it would run on the main thread.
  iank: You can receive a bunch of elements at this point in the
        timeline and then be very explicit what you can do with
        those elements.
  birtles: An example is animation a string, like 1 to 100. For that
           use case touch input and timing wouldn't be sufficient.
  iank: No way to do that in CSS today without using content. And
        that would run on the main thread.

  shane: Coming back to the point about making this look more like
         web animations. Web animations doesn't yet have a
         procedural main thread API.
  shane: This would present to web pages as another version of web
         animations.
  shane: I think the idea of this slotting into the web animations
         framework should be quite doable.
  birtles: Fine to fill in the gap later but we need a story.
  rbyers: Just to make sure I understand the concerns. You also want
          it to be driver by web animations. Do you want the
          animations to be driver by a timeline?
  rbyers: This is in many ways more driven by raf.
  <iank> https://gist.github.com/bfgeek/7710d6283675c4494b92e7e28bc7f6c9
  birtles: My concern is that there is a consistence problem.
           Timelines can be raf driven, maybe that is usable still.
           Want to see how it fits together.
  birtles: Would like to see how it fits in with a declarative API
           as well.

  rbyers: The thing that has always made me uneasy with that is the
          level of abstraction. Fingers on screen as an example.
          Would each finger have a timeline? Feels like an inversion
          of the abstraction level.
  rbyers: Maybe we have a timeline at 60hz and a way to get the
          fingers.
  smfr: Wall clock time is fine but we also have input, having
        different timelines and inputs depending on use case.
  iank: You could expand it so that in addition to the timeline it
        could be fed keyframes and have that drive it?

  iank: Would you prefer to see a web animation API over a CSS
        version of animation worklets?
  birtles: They should all fit together.
  shane: Do you mean you would prefer to see a rough spec of the API
         or prefer to ship a JS API.
  birtles: We should try to do both at the same time, not one or the
           other.
  smfr: I would like to see a web animations v2 draft that animation
        worklets could be built on top of.
  iank: We have gone back and forth a bit internally about which
        order to do these.
  birtles: We have only shipped a small subset of web animations so
           far, still room to change for level one to allow these
           things.

  Rossen: Quick time check. Time for a break soon.
  Rossen: A lot of feedback about a strawman about web animations vs
          animation worklets. Makes sense and we have the right
          people for this. Was there anything else we want to go
          though?
  * fantasai in favor of investigating declarative solutions in
             addition to JS ones, as a general principle
  rbyers: Specific AI is to hash up an explainer this week.
  iank: Syncing properties, running inside web animations. Things
        like that might need more discussion.

  <br len="10min">
Received on Monday, 15 May 2017 08:42:04 UTC

This archive was generated by hypermail 2.3.1 : Monday, 15 May 2017 08:42:05 UTC