[Houdini] Minutes Toronto F2F 2019-06-07 Part II: Paint API, Typed OM, Animation Worklet

=================================================
   These are the official Houdini Task Force
     minutes. Unless you're correcting the
      minutes, please respond by starting
 a new thread with an appropriate subject line.
=================================================


Paint API
---------

  - RESOLVED: We go with AmeliaBR's suggestion
              (
https://github.com/w3c/css-houdini-drafts/issues/877#issuecomment-487651369
),
              for which input custom properties create edges in the
              custom property graph (Issue #877: Cycle possible using
              inputProperties())

Typed OM
--------

  - RESOLVED: Disable SVG text when in a paint worklet, possibly by
              subsetting to SVG Native (Issue #899: Opaque font P&V
              type for PaintWorklet font rendering)

Animation Worklet
-----------------

  - RESOLVED: Accept majid's proposal to use a state() function's
              existence to detect statefulness, and drop the
              superclass check (Issue #850: Avoid StatefulAnimatior/
              StatelessAnimator superclass and use existence of
              state() function)

===== FULL MINUTES BELOW ======

Agenda: https://github.com/w3c/css-houdini-drafts/wiki/Toronto-F2F-June-2019

Scribe: emilio

Paint API
=========

Cycle possible using inputProperties()
--------------------------------------
  github: https://github.com/w3c/css-houdini-drafts/issues/877

  flackr: We realize that with paint worklet you can specify an image
          as an input property, which can itself be backed by another
          worklet
  flackr: So you can in theory, once we implement the TypedOM <image>,
          you could specify a cycle

  flackr: So there are multiple ways we could go about it
  flackr: We could try to spawn the paint worklet
  flackr: or try to prevent recursion somehow
  TabAtkins: The notion of CSS image generator values is not in any
             spec right now
  TabAtkins: We could separate it
  TabAtkins: We have cycle detection for custom properties
  myles: And then what?
  TabAtkins: You'd get a broken image
  myles: The whole chain?
  TabAtkins: Yeah, probably

  heycam: How do you use the image values in the worklet?
  TabAtkins: You can pass them to drawImage
  AmeliaBR: There are lots of very sensible use cases for reusing the
            paint worklet result from another paint worklet
  AmeliaBR: even the same worklet
  TabAtkins: That's true, so probably we don't want to use cycle
             detection
  TabAtkins: but just depth detection
  AmeliaBR: Cycle detection is fine, but not if you refer to the same
            worklet
  TabAtkins: I don't see the difference
  AmeliaBR: I talk about the case in which says paint(tiling) and
            another that uses paint(tiling) that depends on the first
            output
  TabAtkins: OK, so proper cyclic detection would work
  AmeliaBR: When you have a paint reference you treat them as var()
            references to the input custom properties, and then use
            the regular custom property cycle detection

  myles: Is this something that needs to be spec'd?
  myles: HTML depth is not in any spec
  TabAtkins: CSS defines minimum depth and such though
  dbaron: It's also useful just to document so that people can pay
          attention to it

  emilio: I agree that going for depth, the particular limit needs to
          be in a spec.
  emilio: But the custom property cycle detection is in a spec.
  emilio: So if we eagerly detect cycles, how to detect that is needed.
  emilio: And how to treat it.
  TabAtkins: If we do what AmeliaBR says then it's already in the spec
  * emilio agrees

  RESOLVED: We go with AmeliaBR's suggestion, for which input custom
            properties create edges in the custom property graph

Typed OM
========

Opaque font P&V type for PaintWorklet font rendering
----------------------------------------------------
  github: https://github.com/w3c/css-houdini-drafts/issues/899

  flackr: So while experimenting with PaintWorklet I decided to use
          fonts and the general idea seems to be that we avoid
          everything that relies on the document data
  flackr: I wonder if we could expose a font type so that we could
          resolve the font on the main thread and pass it down to the
          paint worklet
  bkardell: So like a transferable font?
  flackr: I'd expect it to be more like an <image> type
  flackr: I don't expect to be able to do anything with it other than
          drawing

  myles: Why not just passing a string?
  flackr: Because that requires going through the platform code
  myles: Should just be thread-safe
  TabAtkins: We disallow parsers on the worklet
  TabAtkins: Chromium's css parser is not thread-safe
  emilio: The fact that Chromium's CSS parser is not thread safe is a
          Chromium limitation
  myles: I'd be hesitant to add more API surface until there's more
         investigation
  iank: You have a side-channel, you don't know if that font is loaded
  myles: I thought these weren't supposed to have any document data
  myles: You want @font-face then this problem is really hard
  flackr: Limiting it to system font limits the use cases
  myles: I think it'd be a good start

  TabAtkins: If we want to do @font-face you could be able to send
             FontFace objects
  myles: And no fallback.
  bkardell: Can you not just transfer the array of resolved fonts?
  TabAtkins: You can do it yourself
  myles: That's not great
  TabAtkins: Or send an array
  myles: I think we can choose between two worlds, one where you (maybe
         temporarily) only have access to installed fonts, and one
         where the worklet is associated with a document and they
         share the same loading context and there's no need to
         transfer anything
  <myles> http://litherum.blogspot.com/2016/09/font-taxonomy.html

  heycam: I was going to ask with the relationship with the font
          loading API, but if for now there's only going to be
          installed fonts
  heycam: If we decide we want @font-face I think we may want to not
          tie it to the document, and the document would have the same
          FontFaceSet
  hober: Wouldn't that require network access from the worklet?
  heycam: If you want all the font types in the doc then sure, but we
          could limit it for blobs and array buffers and such
  [missed]

  heycam: You can have SVG images with text in it
  <dbaron> heycam was talking about <SVG:text>
  myles: But that's not the same, no subpixel AA, etc.
  TabAtkins: We cannot avoid putting text on worklets at some point

  AmeliaBR: SVG images has similar restrictions where there are no
            font-faces and such and it's isolated from the main
            document rendering
  AmeliaBR: In a way it's not that much different from how paint() is
            supposed to work
  AmeliaBR: If we have the same restrictions on the paint worklet so
            that it it can define a font-face where it doesn't have
            network access
  AmeliaBR: so it is not as different as <text> in an SVG image

  AmeliaBR: Whether that addresses authoring use-cases is another
            question
  AmeliaBR: Saying that it can only use installed fonts doesn't seem
            aligned with paint()
  AmeliaBR: Paint is for graphical effects so you want to have control
            over the fonts you're drawing
  AmeliaBR: If you want textual content it shouldn't be using paint()
            but be content instead

  AmeliaBR: The proposal that flackr made was adding a new typed om
            type for fonts
  AmeliaBR: which I think it's a sensible thing for typed om to be
            able to represent
  AmeliaBR: We could set that from canvas as well
  AmeliaBR: Some other people were talking about FontFace object, but
            fonts is much more complicated, it's a list of fonts with
            fallback, etc.
  AmeliaBR: We should aim for typed om to be able to represent all CSS
            data types like the font shorthand
  AmeliaBR: but that's a different question from text in canvas in
            paint()
  AmeliaBR: I'm not sure where in the cycle we'd prevent you from
            using constructed typed om objects constructed in the
            worklet
  AmeliaBR: Seems like Chromium's parser is a limitation for chrome
            but others disagreed
  <bkardell> AmeliaBR is better articulating the question I was
             attempting to ask myles earlier about the use case -- for
             paint
  iank: That part of the parser may be thread-safe already

  TabAtkins: So a multi-part proposal, I think we can do immediately,
             given that you can put <SVG:text>, I think we could add a
             canvas function to render SVG text
  TabAtkins: You can create an SVG text in the worklet right?
  TabAtkins: So part 1 of the proposal is adding back drawText() to
             canvas
  TabAtkins: Second part of the proposal is that we should see if
             parsing font stuff is still a problem for browsers
             (particularly chrome), and if it is figuring out
             something for that in the short term
  TabAtkins: If it's not then probably just expose it and reconsider
             the decisions
  TabAtkins: about exposing parser things in the canvas
  iank: I think that's already thread-safe in blink
  TabAtkins: Font stuff in particular?
  iank: Yes
  TabAtkins: Part three is: in the medium term we investigate how to
             expose web fonts
  TabAtkins: hober's idea about explicitly telling the worklet what
             fonts to use may work
  myles: Another idea is making FontFaceSet transferable
  TabAtkins: You still need a way to thread-that down, but that could
             be a way

  <AmeliaBR> FYI, working demo of text in a Paint worklet, using
             WebGL: https://github.com/AdaRoseCannon/three-paint

  dbaron: I don't recall why we decide text stuff to not be in there?
  TabAtkins: It was a philosophical reason, we didn't want people to
             do a bunch of text stuff in paint worklet
  Rossen: Also we wanted to look how the measuring APIs would work
  Rossen: before doing that

  dbaron: I do recall some system libraries not being thread-safe
  myles: coretext is not thread-safe, we'd have to rewrite it and
         that's fine probably
  dbaron: We're on multiple platforms
  emilio: I think we use mostly harfbuzz these days, and that can be
          threadsafe
  myles: webkit's font handling is threadsafe too

  dbaron: This makes probably a harder dependency on the idea that
          paintworklet is a list of drawing commands and not a raster
  TabAtkins: Yeah it does

  myles: Why adding drawText if we want to investigate better things
         in the future?
  TabAtkins: It'd be in canvas2d right?
  myles: I guess what I'm saying is that the eventual solution may not
         look like drawText at all
  AmeliaBR: Whatever solution should probably be coordinated with
            canvas api
  myles: Right and we may add a new api to canvas2d
  myles: I think it's a bit too early to say that we need to support
         this API
  TabAtkins: I don't like that there's a hacky way to draw text
  TabAtkins: but not a good way

  emilio: Does the SVG stuff _actually_ work?
  TabAtkins: Not sure, can somebody confirm quickly that it works?
  TabAtkins: Disabling SVG text stuff is also a suitable resolution
  iank: We may be able to do the FontFaceSet stuff
  myles: I think that's a bad design
  TabAtkins: It doesn't give us access to installed fonts
  heycam: Wouldn't you be able to use local sources?
  myles: I think not representing the fact that fonts are a list of
         fallbacks is doing a disservice to users

  myles: I think we should have transferable font objects, but they
         probably shouldn't be FontFace
  myles: should be some sort of thin wrapper over what the font
         shorthand represents
  flackr: I'm hoping that typed om font property can represent that
          wrapper

  TabAtkins: Let's solve this in the medium term so we can all think
             about this properly
  TabAtkins: I want to resolve whether text is allowed at all or not
  heycam: It's plausible that the SVG subsystem doesn't know how to
          disable SVG text in a standalone way
  <AmeliaBR> SVG Native
  myles: SVG Native doesn't allow text so it may be a good fit for that

  heycam: So you said that the original reason to disallow text is to
          avoid inaccessible text, is that no longer a concern?
  heycam: or is it because it works with SVG?
  heycam: Because if it's because of the latter we probably don't want
          to give up
  TabAtkins: It was only one of the factors, others include text is
             hard, and high-quality text needing font metrics

  <AmeliaBR> https://ada.is/three-paint/
  AmeliaBR: You could use webgl too, see ^
  myles: No you can't, you'd need to upload the font as a texture
  TabAtkins: That demo is fake webgl, it's 2d
  TabAtkins: We cannot really avoid it

  TabAtkins: Proposed resolution is to disable text when in a paint
             worklet, possibly falling back to SVG native
  TabAtkins: and long term we want to lift the text restriction
  AmeliaBR: We could point to the SVG-in-OT restrictions

  RESOLVED: Disable SVG text when in a paint worklet, possibly by
            subsetting to SVG Native

Animation Worklet
=================
  Scribe: TabAtkins

Avoid StatefulAnimatior/StatelessAnimator superclass
----------------------------------------------------
  github: https://github.com/w3c/css-houdini-drafts/issues/850

  majidvp: Last year, we made a resolution on how to differentiate
           between stateful and stateless animator
  majidvp: Idea was to use two separate classes
  majidvp: You inherit from whichever you want
  majidvp: We looked at some different options: looking at property on
           animator, or length of arguments, etc
  majidvp: While trying to implement this, I realized there were
           complications

  majidvp: So new idea I think addresses the shortcomings
  majidvp: Rather than requiring a superclass to be extended, we look
           at the existence of the state() function
  majidvp: So if your animator implements the state() function,
           they're a stateful animator. If not, they're stateless.
  majidvp: I think that's simpler. The interface matches the contract.
  majidvp: Problem with the superclasses is that it clashes with
           bringing your own class hierarchy.
  majidvp: It'll cause people to just extend StatefulAnimator at the
           top of their hierarchy, which defeats the point.
  majidvp: This seems more consistent with other worklets too -
           PaintWorklet and LayoutWorklet don't require extending a
           class.

  AmeliaBR: An issue I see with that is that from a JS perspective,
            you can add a getter function on an instance.
  AmeliaBR: I'm assuming you judge it at the registration stage.
  AmeliaBR: What if you put one on the instance?
  majidvp: Verify it at registration stage, like paint functions works.
  majidvp: If you add state() later, it'll be ignored.
  majidvp: We want it to be a single value at registration for the
           whole life.
  TabAtkins: So like we'll do a Get on the class object, seeing if
             "state" returns a function, and make the decision based
             on that. This is similar to, say, Promises and .then()
  AmeliaBR: I see some different examples, a state getter and a state
            method. So it's the method you're discussing?
  majidvp: Yes

  <heycam> https://tc39.es/ecma262/#sec-ordinaryhasinstance
  heycam: In the first issue comment, you say the current superclass
          model we have to ask TC39 to add an operation, but I think
          it already exists.
  majidvp: I started looking at this because I wasn't sure how to spec
           it, and AudioWorklet already does this, but they haven't
           correctly specced it.
  majidvp: During that my new idea came up, and I think regardless of
           speccing, this is a better API design.
  <AmeliaBR> ship it. But, you know, spec it first.

  proposed resolution: accept majid's proposal to use a state()
      function's existence to detect statefulness, and drop the
      superclass check

  RESOLVED: Accept majid's proposal to use a state() function's
            existence to detect statefulness, and drop the superclass
            check

  <br type=lunch>

Received on Thursday, 11 July 2019 23:13:23 UTC