W3C home > Mailing lists > Public > public-houdini@w3.org > May 2017

[Houdini] Minutes Tokyo F2F 2017-04-18 Part III: Paint API, Properties & Values

From: Dael Jackson <daelcss@gmail.com>
Date: Mon, 15 May 2017 04:41:39 -0400
Message-ID: <CADhPm3vn7LBsQ_Vd+w8H7sL4YpJyZmi2wXjNX2fG=RxH39J20g@mail.gmail.com>
To: public-houdini@w3.org
Cc: www-style@w3.org
   These are the official Houdini Task Force
     minutes. Unless you're correcting the
      minutes, please respond by starting
 a new thread with an appropriate subject line.

Paint API

  - The group didn't have a firm preference on issue 348 (consider
      making Path2D the only way to render paths with
      PaintRenderingContext2D) but they will search out author
      feedback and then add it to the Paris agenda.
  - RESOLVED: Don't expose constructor and parse methods
  - Issue 326 (Scope of paint-valid flag being box (not box+use/
      size) seems wrong) will be discussed by interested parties.
  - RESOLVED: Any uses of CSS font-relative units without an element
              context should resolve as they do for MediaQueries
  - RESOLVED: add CanvasFilters to PaintRenderingContext2D, but
              supporting only CSS filter functions and not url()
              references to SVG

Properties & Values

  - RESOLVED: Typed custom properties that are URLs resolve relative
              to the file they're in, unlike untyped custom
              properties which resolve where they're substituted.


Agenda: https://github.com/w3c/css-houdini-drafts/wiki/Tokyo-F2F-April-18-2017

Scribe: fantasai

Properties and Values Implementation Feedback

  shane: We made some changes at the last F2F which simplified
         dealing with error cases and retroactive registration etc.
  shane: With those changes, API was easy to implement on top of
         existing custom properties implementation.
  shane: Only issue is how to deal with relative URLs.
  shane: We resolved on dealing with them in one particular way
  shane: for untyped properties.
  shane: Can't remember details.
  shane: That turned out to be quite painful for typed properties.
  Rossen: Why?
  iank: Registered property impl in Gecko?
  dbaron: Don't think it ever landed.

Paint API

  iank: Transitioning custom property that's used in custom paint.
  iank: all in the github repo
  [iank demos stuff]
  <nainar> https://github.com/w3c/css-houdini-drafts/tree/master/css-paint-api/chat
           currently I assume.
  <dbaron> for the future, maybe better to refer to
  <fantasai> dbaron++

Issue 348: Consider making Path2D the only way to render paths with

  <TabAtkins> Topic: https://github.com/w3c/css-houdini-drafts/issues/348
  iank: This was about subsetting paint rendering context2d even
        more and remove canvas part APIs.
  iank: This came from someone at Firefox.
  iank: Removing this would force devs to remove new createPath API
        on the canvas and then stash these paths between frames.
  iank: This provides some optimizations for rendering engines.

  iank: My sense is that we don't want to do this
  iank: For the V1 api
  iank: and instead have a displayList like context, which we've
        talked about previously, for a V2.
  dbaron: My understanding of V1 was that you could implement it on
          top of a displayList-like context
  dbaron: i.e. a canvas that is backed by a display list instead of
          backed by a bitmap.
  iank: What this allows you to do is cache display lists of 2d path
  dino: What he's saying is that if everyone used the canvasPath
        interface, then a GPU renderer could cache the tessellated
        vertexes and store them on the GPU and would not have to
        recreate them every time.
  ChrisL: Another advantage of displayList is that it's much easier
          to make an a11y tree.
  dino: Either way there's still a displayList.
  dino: If we restrict the API, it could make certain drawing
        implementations faster.
  iank: v1 is still implementable using a display list, this is how
        our impl works.
  iank: The other main thing is that we're trying to make this thing
        reusable by canvas AIs, and the ones I looked at heavily
        make use of the path apis.
  iank: If we want a more optimized display list rendering context,
        that should separate API.

  smfr: So proposal is that we allow both the old style path
        rendering to context and the new path 2d stuff?
  iank: Currently that's what we have.
  iank: This proposal wants to remove the context stuff.
  smfr: We're going to keep old style and new style path drawing
        functions on context.

  jack: Are ppl going to use 3rd party libraries that are already
        using these, and this is why we're keeping them?
  jack: And we're assuming they aren't going to break due to missing
  iank: ...
  iank: It's pretty reasonable to make it work.
  iank: Could re-discuss with nical in Paris, but should resolve to
        keep for now.
  dino: End result should be able to do it with canvas and paint

  dino: New version isn't very useful, path objects sit by
        themselves, don't know their lifetime
  dino: Best way is SVG where you explicitly say you're going to
        reuse that object
  jack: I can ping relevant webrender ppl on this bug.
  jack: Saw you mentioned some workarounds
  jack: Seemed reasonable to me.
  jack: He also said no reason to cut this API, but no reason to
        wait until August, could ping ppl now.
  Rossen: So we don't need a resolution now, can just put on agenda
          for Paris and keep working in github.
  Rossen: Doesn't require any changes to current spec anyway.

  <dbaron> (The argument that there are a lot of libraries that will
           just work out of the box with the old path syntax seems
           pretty reasonable.)

Issue 238: Should constructors / parsers be exposed in Worklets?

  <iank> Github topic: https://github.com/w3c/css-houdini-drafts/issues/238
  iank: This is what API we should expose inside paint worklets
  iank: Should the paths api and constructors be exposed or not?
  iank: One side of why shouldn't be exposed is should be getting
        these objects through the style map as part of your input,
        and this would be the same for layout animation as well.
  iank: Flip side is might want to construct these values out of
        other inputs, and then convert it to something else at some
  iank: Thoughts?

  shane: ... gradients, those are quite useful, so at least
         procedure constructors should be usable.
  iank: So want to cssimage parse linear-gradient blah?
  iank: Yeah, can see that.
  shane: Tab responded also.
  iank: Trying to think about additional overhead of exposing...
  iank: ... is the valid case.
  shane: Eventually font face information will have use cases too.
  iank: For font face you'll always want to comment in through
        style, because will need to know about loading font
  iank: Same for images.
  shane: If you're building a painter that wants to use text, and
         you want to use a font.
  shane: Drawing text is a valid operation, so if you always want to
         use Comic Sans, then it's kind of the analog to ?

  iank: Upside to not having this is it may make instance
        construction a bit faster.
  iank: Shouldn't be too concerned.
  iank: Any thoughts on this?
  till: Doesn't seem like there are any real use cases for having
  till: So while perf concerns are slim that maybe should be
        nonexistent use case.
  iank: Can always add them later as well.
  shane: Are you suggesting to just add image value ones?
  shane: I think image value use case is a significant use case.
  shane: If you have a gradient background, makes sense to construct
         that value from within the paint worker
  iank: ... cross-fade() would have to receive two images and
        receive those images to cross-fade them.
  shane: Can you implement cross-fade with compositor operations?
  iank: Yes, very easily.

  iank: Alright so sounds like we'll not expose the constructors and
        pass methods on the typed OM for the paint API, and probably
        layout as well
  iank: but we'll want these for the animation worklet API, because
        we need to set style.
  Rossen: Any other comments?
  iank: We'll have to work out what to do for typed OM, but that's
        separate issue.

  RESOLVED: Don't expose constructor and parse methods

  <majidvp> AnimationWorklet may need them to be available as it
            writes to outputStyleMap.

Issue 326: Scope of paint-valid flag being box (not box+use/size)
    seems wrong

  iank: I want to remove this line here “When the
  <dbaron> from https://drafts.css-houdini.org/css-paint-api/#paint-invalidation
           iank wants to remove: When the size (as determined by
           layout) of a box changes, each <paint()> function’s paint
           valid flag should be set to paint-invalid.
  <dbaron> ... related to https://github.com/w3c/css-houdini-drafts/issues/326
  iank: I think this was related to dbaron's issue
  iank: which is the invalid flag scoping is wrong.
  iank: Not invalid if the box size changed, but not necessarily
  iank: This can just work through other CSS machinery, so don't
        have to explicitly invalidate.
  iank: It will just call this function when it needs to.
  iank: So I think just by removing this line here, would resolve
        this issue.

  Rossen: When we added this, what was the main reason. Was it
  iank: This was in the spec from day 1.
  Rossen: From what I recall, this was there to capture the cases
          where you have to repaint?
  iank: Paint thing is there if it's in the viewport ... you have to
        paint it.
  iank: If box size changes anyway, then UA will repaint it anyway.
  dbaron: The paint() function can be used multiple times per
  dbaron: If box size changes, maybe will need to repaint, maybe not.
  dbaron: What you want to track is whether you're valid for this
          element at this size.
  <dbaron> (I was just summarizing the initial comment of
           https://github.com/w3c/css-houdini-drafts/issues/326 )
  iank: But invalidates always whenever the box changes, so I think
        we can remove that and rely on CSS machinery to call you if
        the concrete size changes.
  smfr: Does the output of the paint have an intrinsic size, as CSS
        understands intrinsic size?
  iank: I think it doesn't have an intrinsic size.
  ChrisL: Why should this wait for later version of the spec? Seems
  iank: Just to keep initial version simpler.

  iank: Does it make sense, then, to rely on the engine to call it?
  dbaron: You should have something that makes the paint valid flag
          invalid when the actual thing you need to paint has
          changed size.
  iank: So I'm saying we .. invalidation, it's at a higher level in
  fantasai: Maybe you just need to change the conditional on that
            line, to refer to changes in concrete size of the
            painted image, rather than changes in the box.
  <dbaron> https://drafts.css-houdini.org/css-paint-api/#drawing-an-image
  dbaron: Might be worth taking this offline.

off-screen canvas

  iank: Some movement in implementations.
  iank: Previous canvas filters is you could do ctx.filter =
        'blur(2px)' we know how to resolve that.
  iank: If you do something like 1em instead of 2px, what does the
        em get resolved against? Similarly to percentage.

  iank: So we said we wanted ctx.filter attribute to accept a
  iank: Still on that some stage.
  iank: Initially to cover these use cases, can do what off-screen
        canvas, they resolve against the initial style of the
  fantasai: That's what Media Queries resolve against.
  iank: They resolve ems against 10px and rems against 16px.
  fantasai: what????
  iank: What? Why? Maybe I should file a bug...
  iank projects code with these constants.
  fantasai: If there's no element context, then should do what MQ
            does, which I think is also defined in V&U.
  fantasai: If not, file a bug against V&U.
  iank: We will change off-screen canvas to match us, and remove
        that interesting default values...

  RESOLVED: Any uses of CSS font-relative units without an element
            context should resolve as they do for MediaQueries

  dbaron: I have trouble keeping track of which graphical operations
          are hard to do if you're switching more display-list
          stuff. Are filters doable?
  iank: Filters are, but resolving SVG wouldn't be allowed.
  ChrisL: Why not SVG?
  iank: Because it's an implicit side input, don't know when it loads
  iank: Eventually want to set filter with CSS filter value, so we
        know you've put a filter value when that loads, we can call
        the paint function again
  jet: This is about accessing external filters
  jet: because internally we use svg filters
  dbaron: ...
  <iank> We don't want to allow "ctx.filter = url(filter.svg)", and
         em, rem would resolve against what media queries say they
  <iank> This is adding CanvasFilters to PaintRenderingContext2D api.

  <shane> what about data URLs?
  iank: Shane asked about data URLs, are they allowed inline?
  iank: We may want to allow that.
  shane: Can always allow it later.
  iank: We can work out if we can support that.
  shane: Although it's free functionality.
  iank: But for now we'll just resolve this.

  RESOLVED: add CanvasFilters to PaintRenderingContext2D, but
            supporting only CSS filter functions and not url()
            references to SVG

Scribe: dbaron

Canvas text

  iank: Not sure what to do about text; still thinking about it
  iank: ... and I also changed the spec to normatively require 2
        global scopes.
  jack: Last time I recall you were planning on swapping them.
  iank: Not sure about initial policy. One policy is to alternate
        globals for each frame.
  jack: Policy choice here makes a choice about what we're trying to
  iank: At the moment our implementation is just using one; spec now
        requires two.
  jack: I guess it would be useful to allow flexibility in the
        policy, but allowing using one always, or one for multiple
        seconds, then the reason for having the rule in the spec is
        pretty weak. People might put up with the resulting bugs. I
        dunno if we need something in there like a note...
  iank: Other thing we'll try is switching every frame.
  jack: The whole thing is a hack because we can't do the right
  jack: So we're trying to make it as noticeable as possible that
        you've messed something up. People might put up with a bug
        that happens a few times a minute.
  iank: Could do 10 frames one one, then 10 frames on the other, etc.
  jack: There's a limit to how good we can make this.
  iank: Could even do every 60 frames.
  iank: I think initially only affecting visible things, people
        probably won't rely -- expecting to not get called when
        scrolled out of view.
  jack: My comment from earlier may apply here too. If there was a
        way to create something with the properties we're going for
        -- so you could say have to have at least two, unless you
        have another way to solve idempotency.
  iank: Yeah, I'll add a note to worklets saying we may revise if we
        can solve idempotency. Frozen realms would be nice, but not
        sure it'll happen this decade.
  iank: We may also do the thing where we kill one, use one for a
        while, bring one back...
  jack: That made me think this wouldn't be an effective deterrent.
  iank: No, we may use flipping every 10 frames and *also* kill it.
  jack: ok.

  jack: Might be an implementor thing -- like chrome devtools have a
        repaint-every-frame, have a devtools always-kill-every-frame
        to let you know if you're doing something wrong.
  iank: For audio and maybe layout may want a callback when going to
        be killed so can save state.
  ChrisL: That would be helpful, especially of audio worklets.
  till: My main concern is the memory constraints thing -- if that
        translates into "on devices" that could make this
  iank: Initially we wouldn't use that opt-out -- then might use on
        Android with <= 1GB RAM.
  till: Developers mostly not using those devices anyway.
  iank: ... yeah, so risk is low.

  Rossen: That's the end of our agenda.
  iank: I don't have anything for layout; was trying to polish up
        paint and worklets.
  iank: Will try to incorporate changes from Seattle in spec for
        next meeting.
  shane: Properties & values?
  Rossen: Yes, let's go back.

Properties and Values

  shane: we decided a little while back in bug...
  <shane> https://github.com/w3c/csswg-drafts/issues/757
  shane: We decided that they're just streams of tokens until
         incorporated into a url(), so if various parts of URL are
         in different files, question about what URL resolves to.
  shane: For untyped custom properties we decided url resolves
         relative to file where variable is substituted.
  shane: For typed custom properties that's hard to do and doesn't
         make sense
  shane: since typed custom properties are a url when they're
  shane: If used in 2 different files, it can't have the same value
         and yet relatively resolve in those 2 different locations.
  shane: If we want typed custom properties to act like
         untyped, there's no good solution.
  dbaron: Do we need them to?
  shane: Don't think so. Could provide more flexibility,
         specifically, typed URLs resolved where they're defined
         relative to the file they're defined in.

  RESOLVED: Typed custom properties that are URLs resolve relative
            to the file they're in, unlike untyped custom properties
            which resolve where they're substituted.
Received on Monday, 15 May 2017 08:42:46 UTC

This archive was generated by hypermail 2.4.0 : Friday, 17 January 2020 19:53:27 UTC