W3C home > Mailing lists > Public > public-houdini@w3.org > February 2016

[Houdini] Minutes Sydney F2F 2016-01-30 Part III: Worklets, CSS Paint

From: Dael Jackson <daelcss@gmail.com>
Date: Sun, 21 Feb 2016 14:49:11 -0500
Message-ID: <CADhPm3txx3-g3LeXCDqH8x4hbexKEbNAjKNp1DvosHmgBX29Jg@mail.gmail.com>
To: public-houdini@w3.org
Cc: www-style@w3.org
=======================================================
  These are the official Houdini Task Force minutes.

         Unless you're correcting the minutes,
        please respond by starting a new thread
           with an appropriate subject line.
=======================================================


Worklets
--------

  - There needs to be a declarative way to load code into worklets,
      but that method is currently undecided.
      - Both <link> and <script> were suggested, but there wasn't a
          final decision since it wasn't needed for FPWD.
  - Worklets will be separated by purpose and the general
      renderWorklet will be removed.
  - RESOLVED: Reject scripts on error (Issue 51).
  - There's currently an open issue in regards to the
      non-determinism of worklet scopes. The group will revisit the
      topic in six months to see if there have been further
      developments to help reach a conclusion.

CSS Paint
---------

  - unregisterPaint() and all other unregister functions were moved
      to v2 due to complexity.
  - RESOLVED: Remove canvas text drawing styles and canvas text.
  - RESOLVED: Defer OverflowCallback to v2
  - RESOLVED: punt sideloading images and other data.
  - RESOLVED: Keep 'Geometry' name in Paint method callback.
  - RESOLVED: Keep existing Paint name; registerPaint()
  - RESOLVED: Option 1, ignore Paint callbacks for cursors
  - RESOLVED: Do both input properties and input arguments
  - RESOLVED: Things should happen on the same frame (in reference
              to when callbacks run).
  - The text for addressing intrinsic sizes should closely mirror
      CSS Images.

===== FULL MINUTES BELOW ======

Agenda: https://github.com/w3c/css-houdini-drafts/wiki/Sydney-F2F-January-2016

Scribe: jet

Worklets
========

  Rossen: Next up, worklets.
  iank: We need things off main thread
  iank: We need to apply changes that can do off main thread (eg.
        paint)
  iank: We need a sane scripting environment,
  iank: eg. workers.
  iank: Though workers allow too many things eg. timeouts,
  iank: there's a need async API for authors,
  iank: hence worklets.
  iank: API is empty global scope.
  iank: API to main script can import a JS module like this. On main
        window, can import scripts,
  iank: that can do layout or paint hooks.
  iank: Worklets are the main thread interface.
  iank: We want to be able to share worklet scope.
  iank: Worklets aren't constructed by authors.

Loading code into worklets
--------------------------

  iank: There needs to be a declarative way to load code into
        worklets, maybe link tag?
  zcorpan: Module scripts?
  iank: Yes.
  iank: That sits on top of new module script infrastructure.
  iank: It's nice for composition of libraries.
  zcorpan: For modules, why do we need import when ecmascript
           doesn't?
  iank: Like script type=module syntax,
  iank: but still need declarative way to do it.
  iank: <link> makes better sense than <script> for this.
  zcorpan: But <link> has "safe" semantics unlike <script>.
  iank: I'm not sure what's the best way.
  iank: esprehn: We need a way to instantiate script in another
        context,
  tantek: there was a proposal to have that be <link>.
  iank: It's not needed for FPWD

  <tantek> link rel=script in particular in XHTML days. never got
           uptake

Number of worklets
------------------

  [https://github.com/w3c/css-houdini-drafts/issues/98]
  iank: How many worklets do we want?
  iank: The spec now says one generic rendering worklet.
  iank: The other idea is to have multiple worklet scopes and hand
        these out among threads.
  rbyers: Why on main thread, why differentiate worklet count?
  iank: Audio may need different worklet types/count.
  iank: We may need to handle worklet lifetime differently.
  rbyers: If we can't reason about instances, why does it matter?
  esprehn: We should be able to reason about worklet scope (eg.
           paint, layout).

  iank: Do we want to have both paint and layout to go to one type?
  Rossen: Why not?
  Rossen: We don't require that layout happens before/after paint.

  surma: If there's only one generic worklet, can we invoke multiple
         worklets?
  iank: You can inject multiple scripts. UA determines how many to
        spin up.
  iank: There's memory cost with discrete worklet types.
  Rossen: We shouldn't spec that and instead allow UA to set types.
  iank: UA can choose to merge them.
  Rossen: That's an implementation detail.
  rbyers: Paint/layout as separate is up to UA?
  rbyers: Author may need to know which worklet to send scripts to.
  esprehn: It's not OK to not know which is which.
  Rossen: paint/layout/DOM in parallel on one worklet seems
          unworkable.
  rbyers: workletcount != thread count. Multiple scopes should allow
          this.
  iank: You can rip out single renderWorklet and separate out
        worklets.
  iank: You get one empty global scope, no generic execution callback.

  <zcorpan> (<link rel="import"> violates the "link is safe"
            assumption)

  smfr: Can you postMessage?
  iank: No.
  smfr: Layout worklets can detect side effects?
  iank: Yes, in layout but not paint.

Should scripts throw upon import?
---------------------------------

  <Rossen> https://github.com/w3c/css-houdini-drafts/issues/51
  iank: When you import a module, should scripts throw? I think not.
  iank: Let the main thread handle such errors.
  ojan: There's no use cases for avoiding exposing side effects.
  ojan: Is there a use case for throwing in worklet scripts?
  iank: Reject promise on failure?
  esprehn: Yes, should expose failures here.

  RESOLVED: Reject scripts on error (Issue 51).

Security Concerns
-----------------

  iank: There's security implications.
  iank: Should use credentials?
  iank: Do same as workers, in worklets?
  ojan: It's good to minimize differences with workers.
  iank: Should we spec to randomly assign callbacks between worklets?
  iank: The spec says UA decides how many worklet global scopes
        there are.
  iank: So UA internals aren't exposed.
  iank: Spec it so that UA must have at least 2?
  Rossen: How do you verify?
  ojan: If you only have one scope, all scripts assume one for all
        UA's.
  ojan: One way to solve that is to require > 1.
  ojan: It's forward-compatible with multi-threaded implementations
  ojan: It prevents authors using globals and expecting state
        preserved.
  Rossen: Can we expect that state is dropped between calls?
  iank: Read-only for global objects?
  esprehn: That's really hard--so much mutable state.
  rbyers: ServiceWorkers intentionally shut down to avoid similar
          problem.
  ojan: We can kill things in background tabs.

  iank: 2 is probably the best option.
  ojan: The trouble with two is that you require 2x memory etc. by
        default.
  rbyers: Let's not spec 2, so we have flexibility.

  zcorpan: Can't we kill worklets more aggressively in devtools, etc?
  ojan: You could have 2nd worklet scope be lower priority.
  shane: Is it true that worklet scopes can't die in the middle of a
         render?
  iank: No.
  rbyers: We can't do the naive thing and just do one scope.
  iank: We will spec that way.
  esprehn: As a developer, if we rarely share scope in production vs.
           development, hard to debug against.
  rbyers: We need some amount of non-determinism here.
  esprehn: How about there's only ever one, but you can opt into
           parallel behavior?
  plinss: But multithreaded browsers have to emulate single threaded.
  iank: We decided to split out paint/layout as separate worklets.
  shane: 90% use case is parallelizable.
  Rossen: I'd prefer read-only marking of scope data from other
          worklets.

  iank: I've made an open issue: non-determinism of worklet scopes
  iank: Let's revisit in 6 months
  zcorpan: We prevent global scope via cross-origin documents now,
           can we reuse that mechanism for worklet scopes?
  esprehn: We need more implementer experience, let's revisit.

  iank: I'll ask for FPWD for worklets this afternoon.

CSS Paint
=========

unregisterPaint()
-----------------

  iank: There's been a couple changes since Sapporo.
  iank: unregisterPaint() removed.
  iank: It got weird.
  iank: unregisterPaint() function in the middle of a paint callback
        gets crazy.
  iank: Callbacks don't get removed from scope now.
  esprehn: 2 use cases: writing tests & hot reloading of scripts.
  esprehn: We've been punting all unregister functions to v2 to give
           time for definitions.

  smfr: Can you register 2 of the same name?
  iank: No.
  smfr: It seems unregister should happen on main thread so
        callbacks can finish.
  iank: We could spec it that way.
  xidorn: Probably could unload a module from a worklet to get that.
  xidorn: You could record all loaded modules, then just remove one
          and reload.
  iank: That sounds like v2.
  smfr: It seems weird to not have unregister when you have register.
  shane: There's no way to tear things down.
  ojan: We can build Paint without it.
  iank: unregisterPaint() behavior is hard to spec, if not needed.
  ojan: e.g., undo rasterized effects?
  smfr: It prevents libraries from overlay behavior by replacing
        existing registered items.
  shane: That's still possible but trickier.
  <shane> actually my comment only applies for libraries that know
          about each other. If you want a library to progressively
          overlay without coordination then yeah you do need
          unregister

PaintRenderingContext2D
-----------------------

  iank: Next change: HTML spec ripped out into tiny pieces
  iank: See PaintRenderingContext2D in spec
  <Rossen> https://drafts.css-houdini.org/css-paint-api/#2d-rendering-context
  iank: The note derived from Canvas2D API in HTML.
  iank: Is there any reason to disallow text?
  smfr: How about resolved fonts?
  iank: Only resolved fonts are allowed in the worklet.
  iank: The font attribute is a bit magic (just the shorthand).
  iank: Only resolved TypedOM font object allowed for text API.
  Rossen: What about font fallback?
  iank: Let's remove canvas text drawing styles and canvas text.

  RESOLVED: Remove canvas text drawing styles and canvas text.

DrawImage
---------

  iank: DrawImage
  iank: Allow drawing TypedOM images.
  iank: We need to add TypedOM CanvasImageSource equivalent.
  astearns: How do we add stuff back in future versions, eg. text?
  iank: if(context.drawText) is one way.

OverflowCallback
----------------

  iank: There's an open issue: do we jettison OverflowCallback?
  iank: OverflowCallback lets you read computed style and add
        additional paint area for box shadows etc.
  iank: The difficult part is that it's a CSS Image, and need to
        spec what happens to a CSS Image when it grows like that.
  iank: It seems too hard to do that now.
  iank: I'm not sure it belongs in this spec.
  iank: It probably needs to be somewhere other than CSS Image in
        the future

  SteveZ: What's blending/compositing before/after behavior?
  <dbaron> You can do overflow without this with
           border-image-source: paint(foo); border-image-slice: 0%
           fill; border-image-outset: <overflow dimensions go here>
  surma: Does it allow clipping masks?
  ojan: dbaron's CSS seems sufficient now.
  <dbaron> Yeah, what iank is saying about a paint API v2 (that lets
           you take over the whole element, or maybe layers, and
           deal with masking, etc.) sounds like a neat v2.
  smfr: The author shouldn't have to specify overflow for that case.
  ojan: You'd need to call paint callbacks to get that geometry.
  heycam: I agree that CSS Images is insufficient here.
  heycam: e.g., sparkly image button effects,
  iank: or chat bubbles.
  iank: We'll punt OverflowCallback to v2

  RESOLVED: Defer OverflowCallback to v2

Calling registerPaint with the same name more than once should throw
--------------------------------------------------------------------

  iank: Issue 7: Do we care about sideloading images and other data
        into worklets?
  iank: Basically need to use TypedOM instead.
  shane: I don't like sideloading images.
  Rossen: Agreed.
  iank: We'll punt sideloading images and other data.

  dbaron: Can we still load images into Paint callbacks?
  iank: es, using TypedOM.

  RESOLVED: punt sideloading images and other data

Geometry interface bikeshedding
-------------------------------

  iank: "Geometry" interface for width, height. We need a better name.
  Rossen: How about Shape to paint into?
  Rossen: Any polygon.
  gregwhitworth: I like "Geometry"

  RESOLVED: Keep 'Geometry' name in Paint method callback.

  heycam: Use a dictionary instead?

RegisterPaint() or RegisterPaintImage()
---------------------------------------

  iank: open issue: registerPaint() or registerPaintImage()?
  iank: We should probably call it PaintImage for future-proofing.
  smfr: It would be nice to match usage in CSS.
  ojan: Let's stick with what we have here.

  RESOLVED: Keep existing Paint name; registerPaint()

Callbacks for Cursors
---------------------

  iank: Issue: what to do about cursor?
  iank: CSS Images are allowed in cursors.
  iank: Do we let Paint callbacks happen for cursors?
  iank: option 1: Ignore it.
  iank: option 2: Call it once and cache image for cursor.
  iank: option 3: Crazy option; to always call back for animations.

  dbaron: It could be just like an image cursor that you have to
          invalidate yourself.
  gregwhitworth: Let's punt for now.

  RESOLVED: Option 1, ignore Paint callbacks for cursors

  iank: Spec text will state that "invalid image"

  <heycam> If you use paint() in the cursor property, then it's
           always treated as an invalid image, and so fallback to
           the next cursor type after the image.
  <tantek> verified that caret only takes color for now
  <tantek> https://drafts.csswg.org/css-ui/#propdef-caret-color
  <shane> SVG carets: https://commons.wikimedia.org/wiki/File:Carrot.svg

  <br length=20>

Use Input Arguments?
--------------------

  iank: Next issue: currently, Paint uses a list of input properties
        and explicitly invalidate on.
  iank: Do we use input arguments instead?
  iank: The downside is that you can't, for example, offset just
        color property.
  iank: Basically, Paint can accept a bunch of input props when you
        register.
  iank: The problem is you can't reuse the same function elsewhere.
  iank: The solution is to use arguments instead.
  iank: e.g., for linear gradients, you can reuse with different
        arguments.
  iank: The problem is that you can't use var(color), need var(--bar)
        to preserve token stream.
  iank: See issue 100
  <dbaron> https://github.com/w3c/css-houdini-drafts/issues/100
  esprehn: You'd want both input list and input arguments.

  iank: option A. use input props as-is
  iank: option B. input arguments
  iank: option C. use both
  <dbaron> Could also pass renamings rather than positioning
           arguments, e.g., --foo=--quux

  dbaron: I'm not crazy about positional arguments.
  dbaron: I'd rather have named arguments.
  iank: Input properties are slightly easier to use.
  iank: Properties are spec'd for performance reasons.
  iank: You can limit mutable properties.

  surma: How about using input properties list and also allow custom
         variables to be remapped?
  surma: How about using only defined named parameters to avoid
         having 2 ways of passing arguments?
  shane: Properties have implicit dependencies on other properties.
  shane: It's bad to have to specify which arguments you accept each
         time. I think we need both.
  iank: How about input properties for now?
  shane: How about both for now?

  heycam: Input arguments creates CSS functions that aren't like
          other CSS syntaxes e.g., gradients has more ergonomic
          syntax.
  shane: typedOM can allow better ergonomics in the future.
  ojan: No objections to both?

  RESOLVED: Do both input properties and input arguments

When do callbacks run?
----------------------

  iank: Next issue: When do these callbacks run?
  iank: e.g., callback before a frame paints,
  iank: or, allow long running callbacks to smear across frames.
  smfr: There should be synchronization between changing style from
        script and scripts that cause paint callbacks to run.
  dbaron: Agreed
  Rossen: We should be able to handle both.

  RESOLVED: Things should happen on the same frame (in reference to
            when callbacks run).

  iank: next issue: no way to prerender in constructor for reuse in
        multiple frames
  <dbaron> smfr: seems like a special case of getting arbitrary
           images into paint callbacks
  <jet> https://github.com/w3c/css-houdini-drafts/issues/101

Declaring intrinsic sizes
-------------------------

  <astearns> https://github.com/w3c/css-houdini-drafts/issues/31
  iank: Open issue: how to declare intrinsic sizes?
  esprehn: We need intrinsic size to do tiling.
  smfr: Why not use background size?
  esprehn: We can't do that for border image.
  dbaron: You can't tile for border image anyway.

  esprehn: For border image, what intrinsic size do you get?
  ojan: There's 9 pieces to border image.
  esprehn: Also list marker image.
  dbaron: The default is 1em x 1em or something like that.
  <dbaron> https://drafts.csswg.org/css-images/#object-sizing-examples

  esprehn: In v1 do we punt on intrinsic size?
  ojan: Does SVG have interoperability for that case? (no intrinsic
        size?)
  dbaron: Gecko has bugs there, especially border image + SVG
  shane: Can we use border-image width?
  dbaron: Read CSS Images carefully to get spec text.
  dbaron: If CSS Paint needs to define the image it produces as
          having no intrinsic dimensions.
  <dbaron> https://drafts.csswg.org/css-images/#intrinsic-dimensions
  <zcorpan> http://software.hixie.ch/utilities/js/live-dom-viewer/saved/3848
list-style-image with intrinsic-sizeless svg

  shane: We need a reasonable thing for border image.
  shane: There's lots of requests for new border effects.
Received on Sunday, 21 February 2016 19:50:09 UTC

This archive was generated by hypermail 2.4.0 : Friday, 17 January 2020 19:53:26 UTC