W3C home > Mailing lists > Public > public-houdini@w3.org > March 2017

[Houdini] Minutes Seattle F2F 2017-01-10 Part I: CSS Painting API Level 1

From: Dael Jackson <daelcss@gmail.com>
Date: Sun, 5 Mar 2017 10:59:56 -0500
Message-ID: <CADhPm3t2yTSXB3CXR2yM3xM1mSFGMj=yn8uBaNbPQ8fkNkUqhg@mail.gmail.com>
To: "public-houdini@w3.org" <public-houdini@w3.org>
Cc: www-style@w3.org
   These are the official Houdini Task Force
     minutes. Unless you're correcting the
      minutes, please respond by starting
 a new thread with an appropriate subject line.

CSS Painting API Level 1

  - RESOLVED: Move all Houdini APIs previously on window to
  - iank introduced his solution to addressing when multiple worklet
      global scopes are registered with the same name.
      - Text around memory constraints will be added to the spec.
      - There were some lingering concerns, but iank will add the
          edits into the spec for people to review actual language.
  - iank will add links to examples and details from the explainer
      into the spec.
  - Using JS to registerPaint for foo and calling it in one global
      scope but not another will be added to the spec as an error
  - RESOLVED: New WD of Paint after edits


Agenda: https://github.com/w3c/css-houdini-drafts/wiki/Seattle-F2F-Jan-10-2017

  Rossen Atanassov, Microsoft
  Tab Atkins, Google
  David Baron, Mozilla
  Bert Bos, W3C
  Bogdan Brinza, Microsoft
  Emil A Eklund, Google
  Elika Etemad, Invited Expert
  Koji Ishii, Google
  Ian Kilpatrick, Google
  Peter Linss, Invited Expert
  Jack Moffitt, Mozilla
  Simon Pieters, Opera Software
  Naina Raisinghani, Google
  Hiroshi Sakakibara, Beyond Perspective Solutions
  Elliott Sprehn, Google
  Alan Stearns, Adobe
  Shane Stephens, Google
  Surma, Google
  Jet Villegas, Mozilla
  Philip Walton, Google
  Greg Whitworth, Microsoft
  Chris Wilson, Google

Scribe: fantasai

CSS Painting API Level 1

Issue #330: https://github.com/w3c/css-houdini-drafts/issues/330

  iank: From TAG review:
  iank: We added attribute paintWorklet[?], TAG feedback was should
        this be on window.CSS instead of Window.
  iank: There was previously some hesitation to add there, because
         was for utility functions like supports.
  Rossen: What swayed TAG?
  dbaron: General concern of adding things to Window, because also
          pollutes global scope.
  dbaron: It certainly doesn't have to be .CSS
  dbaron: Suggestion was, can this be something other than Window.
  TabAtkins: .CSS sounds find.
  esprehn: If ppl use worklet, won't collide.
  plinss: Maybe something layout related rather than just CSs, to
  fantasai: window.layout?
  plinss: Keep .CSS for css-specific functions.
  iank: Also question of moving registerProperty from properties and
        values spec.
  iank: If we move one, should probably move all of them.
  iank: Not hearing any strong objections to moving all of the
        worklet stuff to .CSS
  fantasai: Just have one point about whether to move to
            .somethingElse rather than .CSS
  zcorpan: These are properties?
  zcorpan: There was a proposal to make .CSS into a webIDL namespace
  zcorpan: Right now only support methods, but could change webIDL
           spec to allow properties.
  TabAtkins: Seems like an easy fix to me.
  iank: Do we move things like registerProperty to CSS namespace as
  fremy: Makes sense to me.
  iank: Any objections?
  Rossen: No

  RESOLVED: Move all Houdini APIs previously on window to window.CSS

  <zcorpan> https://github.com/heycam/webidl/pull/121

Threading issues

  iank: Next issue was new threading behavior for multiple worklet
        global scopes on different threads, potentially.
  iank: Spec is registerPaint and draw??
  iank: Summary of what happens how...
  iank: Every time you register paint function against global scope,
  iank: can happen in multiple global scopes.
  iank: Each time you do that, potential for two different global
        scopes to register subtly different classes under same name
  iank: e.g. possible for someone to register a paint class under
        the name foo, possible that it can have two different class
        structures in the different global scopes, e.g. one
        invalidates on property one, and one invalidates on property
  iank: Behavior here is that once inside of the global scope, we've
        registered, created a new definition. We queue a task
        function on the document, which will then insert into the
  iank: This checks if the input property exists on that map. If it
        does, checks if they are exactly the same.
  iank: If they don't match, sets that entry to invalid.
  iank: If you try to use that name now, it gets treated as invalid
        because you tried to register two different things under the
        same name.

  fremy: Why allow this at all?
  fremy: Why not restart the worker?
  iank: Can't do that, b/c on different threads, so racy.
  iank: Need to have an essential truth, which is the document's
  iank: If you wanted to do that, you'd have to acquire a lock.
  fremy: Could write a promise.
  fremy: When you register, you get a promise. If it registers, it
  iank: It's still racy.

  iank: I think this is pretty simple.
  iank: Might get a case where valid where previously invalid.
  iank: This mainly impacts Firefox and ?
  jack: We have some concerns about worklet multiple scopes thing.
  jack: Simon filed a bug asking for normative language asking for
        this to be ?
  jack: As someone ships an impl that allows a single scope ...
  iank: We have a bug I haven't fixed, we need to add to spec threat
        there needs to be at least two global scopes per
        implementation unless there's memory pressure.
  <astearns> Issue iank mentioned:
  jack: Concern about that solution is that it's a hacky way around
        the problem, which is that you don't want the worklets to be
        able to pollute the scope.
  jack: Is there some other way to achieve without doing this round
  jack: So ppl don't depend on ?
  iank: Other way to do this is do something like freezing the
        globals inside of the worklet global scope.
  iank: I don't think that's a good situation, doesn't really solve
        ? problem
  iank: ...
  jack: Can't ensure that each worklet, when it starts executing,
        has a fresh scope.
  esprehn: No way to do that. JS is the wrong language for that.
  esprehn: Also hostile to polyfilling.
  jack: ...
  esprehn: Oh, so you start/stop the whole system each time?
  jack: If some worklet polyfills in a scope, then another worklet
        runs in that same scope, has a bunch of polyfills it didn't
  esprehn: Start/stop cost and memory overhead is to much for that.
  jack: If this becomes a web compat issue, that they have to run in
        the same scope, will prevent any parallelism possible in
        Servo in the future.
  esprehn: Shouldn't be as bad as it seems.
  iank: Added many things to mitigate that.
  iank: All run as modern scripts instead of classic scripts, so
        don't have access to the self attribute, similar to workers
  iank: which allows you to access the globals.
  iank: It's possible to stash stuff on the Array prototype
  iank: but I think that having ? normally applying, at least two
        global scopes is the best we can do.

  jack: Except for memory pressure...is that just gonna be wording
        in spec?
  jack: Is it expected that Android version will always have one?
  iank: Something with 512MB of memory, e.g.
  iank: Does that address concerns?
  jack: I'd say we're still concerned, but will wait to review spec.
  iank: [gives sample wording]

  jack: Simon's bug also mentioned, need a way to pass state to ?
  jack: The issue we want to prevent is stashing state in global
        scope, then depending that that's there the next time you run
  jack: But e.g. animation worklet needs to be able to stash state,
        will there be a special API?
  iank: Killing an instance of paint class and restoring it should
        be okay, don't expect much state.
  <dbaron> what's being shown on the screen is the end of
  iank: Expect that people in constructor will build up a canvas ?
  iank: Inside the animation worklet, want state from frame to
        frame, which is not true for paint or layout.
  iank: Many ways to solve, maybe callback "we're going to kill
        global scope, save state into this object, when we construct
        again will bas this object in"
  iank: That is a possible mechanism for this.
  iank: Custom properties was mentioned as a mechanism, I don't
        think that would work.
  jack: Don't have a particular preference, as long as there is a
        way to do it so that people don't do crazy things.
  iank: We'll see once we ship if ppl stash state.
  iank: I don't think they'll do for paint or layout.
  iank: Might do some caching inside structure, that's about it.
  iank: Does that match your intuition of what devs might do?

  dbaron: One question I have about the thing you're inserting:
  dbaron: What normally happens when you register two different
          things under the same name, and why is one piece of it
  iank: Previously, the spec language was, at the very top it would
        check entering the properties map ...
  iank: Was wrong, because if two were going one would fail.
  iank: This registers everything on the global scope, does a
        cross-thread jump to the document.
  iank: And basically does a check. Gonna try to add its set of
        painting input properties to the map
  iank: If they're different, will nullify that class name.
  iank: You tried to register "foo" with two different input.
  dbaron: Doesn't there need to be some other global thing that says
          how to paint a particular image?
  iank: That's per-worklet scope.
  dbaron: So if you have, e.g. background: paint(something), you go
          ask multiple worklet global scopes?
  iank: What you do is, you do a bunch of work on the document to
        see if that's a valid ?
  iank: Grab the input properties map
  iank: If it doesn't exist, it's invalid.
  iank: If it has the invalid flag, treat it as invalid image.
  iank: At that stage select a WorkletGlobalScope.
  iank: Then do a cross-thread jump, and then painting on that
  dbaron: Partly I haven't really seen examples with the whole thing
  dbaron: Last time I looked in spec, didn't see those.
  dbaron: Wasn't clear, what makes that.
  dbaron: I just don't understand how this fits together and it
          still doesn't make sense.
  dbaron: If you're saying the identifier that you're registering is
          registered from paintWorklet, what makes that paintWorklet
          exist in the first place.
  iank: Up to the UA.
  iank: UA must have at least two running.
  iank: When you import this function it, makes those local scopes
  iank: When you call import on the script, it will import that
        script into the two worklets.
  dbaron: And then that script will call register.
  iank: Yes.
  iank: Language to ensure global scopes exist, and then run.
  dbaron: Feels weird to me that registration happens inside worklet
          rather than outside.
  iank: I don't think you can do it a different way, since it has to
        be local to ?
  iank: You can't register class in one thread and use it in another.
  iank: If you could it would be amazing!
  <dbaron> Yeah, I agree it doesn't make sense... but it still seems

  smfr: Custom paint is running on separate thread.
  smfr: How does author synch custom paint with other things in the
  iank: On import, once the promise returned from import has
        resolved, the spec language is that ? and everything should
        be set up, and in next frame if you use custom paint with
        that name it should work.
  smfr: Wondering if you can request animation frame, change content
        in the document in the normal way, and also have the paint
        worklet, and have the results show up.
  iank: Since you don't know if there's going to be a network round
        trip when you import, we have to wait until promise resolve.
  smfr: First time, subsequent times can be synchronized.
  iank: Right.
  smfr: So subsequent paints are still happening on the thread,
  smfr: So UA has to waiting until custom paint ?
  iank: Promise will return once all scripts have run.
  iank: That includes doing this potentially post-message, hey
        document, I have registered paint function.
  smfr: you get some small amount of parallelism running the script
  iank: All run synch immediately, but UA can choose ...
  jack: Circle can run each iframe already its own separate thing,
        not in the same context.
  jack: This means you have multiple things that might be
        generating, painting...
  jack: already happens in Servo.
  iank: We're doing similar thing with out-of-process iframe.

  jack: Notice that this paint in particular, doesn't have much in
        the way of motivating examples. Some small examples in the
        end, like painting single conical gradient. Is there going
        to be more?
  iank: Explainer in github
  <astearns> https://github.com/w3c/css-houdini-drafts/blob/master/css-paint-api/EXPLAINER.md
  fantasai: Any reason not to have that in the spec?
  iank: Could pull it into spec.
  jack: Get a link to spec, read it, seems okay but what is it for?
  astearns: Explainer doesn't seem to have that much more in terms
            of motivating examples.

  jack: Might be interesting to know what is the planned use for
  jack: What feature were you trying to add to gmail that lead you
        down this path.
  esprehn: Ink drop effect in material design,
  esprehn: The puddle effect,
  esprehn: people do crazy things to make that work.
  esprehn: You want to just specify center of ripple.
  esprehn: Currently do crazy things with expanding divs
  <astearns> codepen for inkdrop-ish effect:
  jack: So you'd register paint worklet on the button, but then
        don't have mouse coords
  esprehn: JS has that info,
  esprehn: Because input of painting takes variable.
  esprehn: You can do background-image: paint(ripple) and then use
           vars to pass things into painting.
  TabAtkins: We also added arguments.
  iank: I can add it to the spec.
  fantasai: Just put it all into the spec directory, even if not
            inlined into main spec HTML file, can be linked from it
            and all published together as a package that explains
  iank: We were seeing ppl dumping huge amounts of DOM to get these
  iank: I'll add that to explainer

  iank: Final part of thread-hopping thing
  iank: is once you select a worklet global scope to invoke paint
        function on, that workeletGlobalScope may never have seen
        that registered class, in which case it does ? back to
        document, saying this thing is now invalid.
  iank: Might have a racy thing where it works once but not
  iank: I think it's fine, something author shouldn't be doing
  dbaron: What is it author did wrong there?
  iank: Author did is in JS that registerPaint for foo didn't get
        called in one global scope, but did in another.
  iank: Worklet global scope that was used for painting may not have
        seen foo.
  dbaron: E.g. if they imported multiple scripts and they loaded in
          different orders in different worklets?
  iank: No, worklets spec always loads in same order. Did something
        nasty, they imported one script, a paint function got called
        that set a global variable on Array, and a subsequent script
        checked var on that array, but then didn't register foo.
  iank: Will work on one frame potentially, but not subsequently.
  jack: What is the error behavior here?
  jack: Seems like something to prevent in general.
  jack: Does the spec say UA should issue a warning?
  iank: Yeah, that's a good point.
  ACTION: iank make this case an error
  <RRSAgent> records action 1


  iank: ... namespaces
  iank: A few minor edits.
  iank: Other major addition to spec is input arguments.
  TabAtkins: Paint function can pass arguments, instead of having to
             rely on variables. Can use more than one instance of
             the paint function.
  plinss: From CSS?
  TabAtkins: Yeah.
  iank: We resolved basically this.
  <iank> https://github.com/w3c/css-houdini-drafts/issues/219
  iank: There's a new static getter which describes exactly the
        syntax in the properties and value spec.
  iank: can have <length>, <color>, etc.
  iank: Additional input arguments.

Moving toward publication

  Rossen: That's all on paint?
  iank: Yep.
  Rossen: So republish once edits are in?
  iank: Yep
  TabAtkins: Because arguing about it in PR.

  TabAtkins: Way you wrote several arguments does very explicit type
  TabAtkins: I suggested doing more webIDL like thing where it
             coerces more.
  TabAtkins: e.g. can coerce into a string instead of type-checking
             and throwing out.
  iank: I converted e.g. alphaValue
  iank: This basically copies what custom elements is now doing.
  iank: I converted to infra maps.
  jack: We spend last year in Servo building web render
  jack: This is absolute wost case per for us, using canvas2D with
        custom paint functions.
  jack: Understand it's very pragmatic, but hoping that in future
        there's a version of API that's easier to optimize to GPUs
  alex: Would be nice to have that as a change request to canvas,
        would affect all other apis.
  jack: Is there a draft spec?
  iank: Can you write one? :)

  dbaron: This need to be as bad as you think.
  dbaron: This is using a subset of canvas API that doesn't have any
          readback mechanisms.
  dbaron: You don't need to implement in terms of a bitmap.
  jack: That's one issue
  jack: But when you do things like [...]
  jack: Batch operations where 1st & 3rd require a draw ? and 2nd
        requires another draw ?
  jack: modified version of canvas API, same calls, but not order
  jack: Able to optimize that, regardless of pixel readback.
  dbaron: Don't see how you can't do that.
  jack: Canvas is order-dependent.
  esprehn: Can still build a graph underneath that.
  dbaron: Don't see how it's worse than adding objects to a scene
  jack: Ok, haven't done any investigation of that.
  dbaron: Have to think about whether we want to ship a Gecko
          version of this using bitmap canvas, or to wait until we
          have something better than bitmap canvas to ship
  iank: ...
  iank: we already have all that machinery for canvas anyway.

  jack: Is the ink drop effect reproducible with this version of
  iank: Yes.
  iank: There's 2-3 custom properties, one which is for xy mouse
  iank: one is for radius of ink drop effect.
  iank: Set CSS animation on radius to go from 50 to 100 or whatever
  iank: and then custom paint just draws circles relative to xy and
  iank: Actual material design spec is more complicated.
  iank: If got multiple button clicks, could have multiple ripples.
  iank: Also if you move your finger around while held down, will
        move around ripples.
  iank: Another example is in gmail-
  iank: E.g. this icon here is 20-30 elements.
  iank: You could imagine this is just one custom paint, it's a
        string of images to paint.
  iank: Suddenly this swings down to O(N) insted of O(30*N).

  iank: This point you raised here, wrt referencing canvas api:
  (you = Alex)
  alex: process issue, not really sure TAG cares.
  alex: Common with things that have an HTML hook.
  alex: How far along is HTMl5.1, 5.2, etc.

  iank: Other thing to discuss is canvas filters and canvasText.
  iank: Wanted to think more about how it fits into typed OM, so
        will come in a later WD once we've sorted that out.
  dbaron: Text might also have threading issues, depending whether
          system's text APIs can run in multiple threads.
  iank: canvasText has a font attribute, which when you set it
        instantly does a cross-thread jump to check if font is
        loaded or not, and can potentially fail.
  iank: Instead what we want to do is have a custom properties which
        is a resolved font face.
  iank: That's a resource in terms of typed OM, can be loaded, can
        be ready.
  iank: That means if font face changes state changes at any point,
        know it's ready to use.
  iank: Instead of accepting a string, will accept fontFace values.
  iank: Similarly for filters, cuz can reference SVG.
  iank: Need to disable that, because that has same loading issue.
  iank: So if we want to support SVG portion of canvas filters, need
        to add that through typed OM.
  <dbaron> there was something there about the resource state
           changing meaning that there's a change in the typed OM
           value which means stuff gets called again.

  iank: Should we resolve to publish another WD after edits?

  RESOLVED: New WD of Paint after edits

  <br duration=15m>

<div id="DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2"><br />
<table style="border-top: 1px solid #D3D4DE;">
        <td style="width: 55px; padding-top: 13px;"><a
width="46" height="29" style="width: 46px; height: 29px;" /></a></td>
		<td style="width: 470px; padding-top: 12px; color: #41424e;
font-size: 13px; font-family: Arial, Helvetica, sans-serif;
line-height: 18px;">Virus-free. <a
target="_blank" style="color: #4453ea;">www.avast.com</a>
</table><a href="#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2" width="1"
Received on Sunday, 5 March 2017 16:07:23 UTC

This archive was generated by hypermail 2.3.1 : Sunday, 5 March 2017 16:07:24 UTC