W3C home > Mailing lists > Public > public-houdini@w3.org > December 2018

[Houdini] Minutes Lyon F2F 2018-10-25 Part III: Font Metrics, CSS Parser, Animation Worklet

From: Dael Jackson <daelcss@gmail.com>
Date: Wed, 26 Dec 2018 17:51:53 -0500
Message-ID: <CADhPm3v5O95voW+u2S_ccOfU7yUGhp9C-ynbZ4BVB-RnY9Sv7A@mail.gmail.com>
To: public-houdini@w3.org
Cc: CSS WG <w3c-css-wg@w3.org>
   These are the official Houdini Task Force
     minutes. Unless you're correcting the
      minutes, please respond by starting
 a new thread with an appropriate subject line.

Font Metrics

  - A new proposal for font metrics of each character was presented by
      koji for feedback.
      - Initially the spec will define separate at caret position, not
          grapheme cluster. In later iterations of the spec a new
          property may be added for grapheme.
      - Originally the proposal said to use the logical order of
          characters, but after discussion it was decided that it
          would be better to use visual order, and add number of code
          units for each TextMetric unit.
      - This spec/proposal needs to stay aligned with SVG glyph

CSS Parser

  - There's interest in exposing the lexer, but that was scoped into a
      later version of the spec since there were several dependencies
      to do it, including that browsers are still working on aligning
      their parsers.

Animation Worklet

  - Chrome has done a lot of work implementing and improving the
      Worklet including moving it off the main thread, differentiating
      between stateful and stateless animations, and impacting all
      properties (not just fast-path ones)
  - majidvp reviewed the new proposal for timelines and starting with
      pointer inputs to try and develop an approach for animations
      based on interactions. (Issue #2493:
  - RESOLVED: Make Animation Worklet API as FPWD

Properties and Values API

  - RESOLVED: Add declarative registration into properties and values
              API (Issue #137)


Agenda: https://github.com/w3c/css-houdini-drafts/wiki/TPAC-F2F-October-2018

Scribe: TabAtkins

Font Metrics

Revised proposal of font metrics for each character
  github: https://github.com/w3c/css-houdini-drafts/issues/828

  koji: This is about issue 828
  koji: Request from authors that want character advance information
        for each character of a string
  koji: In canvas API we once tried it, but it had lots of feedback,
        so this is a revised proposal.
  koji: use-case is an author with a string, they want to know caret
        position for drawing between each pair of characters
  koji: Or decorations on specific characters
  koji: This revised proposal has a .character FrozenArray with
        TextMetrics, a new interface we're defining.
  koji: Each TextMetric has metrics for one grapheme cluster
  koji: And has an index into the original string, by code unit
  koji: Has advance, and a boolean indicating rtl vs ltr
  koji: We've gotten some feedback on it already.
  koji: From Myles.

  myles: When I read this I thought it was about caret positions, not
         grapheme clusters.
  myles: So is it one entry per grapheme cluster, or one per caret
  koji: Ambiguous. Authors I talked to didn't understand the
  koji: You're right there's some subtle differences between those.
  koji: As far as I understand the request, they often want the
        character, to draw decorations.
  koji: Most webapps handle grapheme cluster as the minimum unit to
        apply stuff to.
  myles: It should be noted that there's a jquery plugin to get caret
  myles: It inserts spans into the content and then gets client rects.
  koji: I think in the long run, authors might want to different
        distinctions for those two. But for initial level, start with
        caret position.

  myles: So why not specific how ligatures work?
  myles: You say "the UA *may* produce one cluster for a ligature"
  koji: Good point.
  koji: Intention was, as far as we could tell, impls do slightly
        different things with caret positions there.
  koji: There was feedback from someone else preferring us to say that
        it should match UA behavior.
  myles: I think that's reasonable.
  myles: If intention is to draw a background on a string, and it has
         an ffi
  myles: And if they want to draw it just behind the f...
  dbaron: It seems like every UA has some behavior for carets in the
          middle of a ligature.
  dbaron: I hope there's no UAs that totally put it off on one side.
  dbaron: But even if the behavior is different, it still seems we
          could expose what UAs do in that case.
  dbaron: So you'd have a more interoperable behavior for the number
          of *entries* the dev would see in the array.
  myles: Right. We shouldn't *fully* specify because in some
         situations we divide ligature evenly by number of grapheme
         clusters, but that's not great. We do have a native API to
         give us correct boundaries, we're just not using it. I'd like
         the flexibility to get that.
  koji: Right, like I said earlier, the return value should match what
        the UA does.
  dbaron: Right. If you do "fix", I don't want some UAs to return two
          entries, and other get three, just because some don't
          provide inter-ligature information.

  fremy: If you have emojis that are composed of multiple chars, this
         API then doesn't work.
  myles: If your string is "e<family-emoji>", the result is two
         entries. First is the letter "e", second is the multi-char
         family emoji.
  dbaron: And same for e + combining-acute-accent. Those aren't
          treated like ligatures.
  myles: The number of entries in the result is not font-dependent, is
         what's important here.

  TabAtkins: What about regional-indicators? (flags)
  myles: If your font doesn't support flags, you get one grapheme
         cluster, it just looks like a pair of characters.

  <koji> The next feedback to discuss is "Since the clusters will be
         in visual order, we should determine if It’s in the direction
         of the base direction or if it’s always LTR. (Internally,
         WebKit always uses LTR, and if the base direction is RTL we
         do some processing to flip it around so our internal
         visual-order data structures are always LTR.) I’m not arguing
         for one or the other; just that we need to specify which way
         it is."
  myles: This should be visual order, right?
  koji: Request is to determine advance of source string.
  koji: So in my proposal the char is in logical order, not visual.
  myles: I thought you said a use-case was putting a background behind
         part of the string, how do you do that if it's in logical
  koji: By making chars in logical order, author can determine where
        each character is, then the author can process it themselves.
  myles: Does that mean at a fragment boundary you could get a really
         big negative advance?
  fremy: This was part of my feedback as well.
  fremy: Is the advance negative in that case?
  fremy: If you want logical order, this needs to break across bidi,
         or use visual order.
  myles: The JS i18n APIs would probably be interested in adding some
         APIs for this if you really want it in logical order. But I
         think it would be best in visual order.
  koji: This interface has ltr vs rtl, so author can control this
        somewhat themselves.

  TabAtkins: You need to know how many chars you're formatting to do
             visual order, right?
  myles: Right, you can only reasonably call this *after*
  koji: So the consensus is to use visual order, and add number of
        code units for each TextMetric unit.
  myles: You may want both "codeUnitIndex" *and* "lengthOfCluster",
         since it's in visual order.
  <heycam> with https://drafts.css-houdini.org/font-metrics-api/#measure-api

  myles: Most important question I have is how you associate this call
         with a font.
  fremy: There is the measureText function in the canvas api
  myles: Is this a new thing, or a replacement?
  koji: We want to sync this with the canvas api.
  koji: We'll port this to canvas api once we agree on it.
  heycam: So the FontMetrics API spec has a new, separate measureText
  koji: Proposal is to add .characters to both FontMetrics and Canvas
  fremy: So this is a mixin that will be used in both interfaces?
  koji: Yes.

  myles: Next feedback - unsure if this makes sense to run on an
         arbitrary element, since arbitrary elements can have children?
  heycam: That's my question, yeah - what does the index count into?
          What about display:none? etc
  heycam: I think there are similar index issues with the string API.
          You have whitespace collapsing/trimming. Need precise
          definition of what indexes are used.
  koji: I understand that part isn't defined here. If we applied this
        to element.meausreText(), we have to define that.
  koji: Currently the proposal only covers measuring a string.
  koji: I'll work on a proposal to define the element case.

  heycam: In SVG we have a silly character-positioning API, and it's
          pretty annoying.
  heycam: If there was a way to avoid all that and just stick to
          strings, that would be nice.
  myles: So I'd like to propose removing the measureElement function.
         Just keep it to strings for now.
  myles: There's more complications, like letter-spacing and such.
  heycam: And text-transform - one character suddenly becomes two
          grapheme clusters, etc.
  myles: Another way to do it is not take StyleMap, but just a small
         set of properties you want to handle, like font-family and
         font-weight. That's what the canvas api does.
  myles: You can't specify font-variation, etc.

  heycam: Ultimately it depends on the use-case.
  heycam: If they want to measure stuff in the DOM, but they can't
          measure everything, maybe not useful.
  eae: Majority of use-cases we've observed are for out-of-dom

  Rossen: So many things we could resolve on, lot of feedback.
  Rossen: I see a request to remove measureElement().
  Rossen: We need to change order to visual.
  Rossen: Add .lengthOfCluster
  heycam: Define how whitespace collapsing, text-transform, etc that
          cause difficult mappings between characters and clusters.
  myles: And change how ligatures and metrics interact.

  krit: SVGWG is also looking at this problem for the counting part.
        At the moment svg1.1 says we should use unicode codepoints,
        that's not very consistent. In our investigation we found
        grapheme clusters aren't well-specified.
  krit: There might be bigger issues.
  myles: Unicode *tries* to specify what grapheme clusters is. If
         that's insufficient, we have larger problems.
  * heycam wonders how Shadow DOM would interact with measureElement,
           if we kept that
  krit: We want there to be alignment between fontmetrics and SVG
        glyph counting.
  myles: We've been talking about a number of things that need to
         change, but in general this is a good direction to go.
  krit: Agree, very useful.
  Rossen: So please take feedback and reflect it into the proposal, we
          can discuss it over the issue in the future.

  myles: One more -
  myles: It seems totally reasonable for an author to want to use this
         api for things like caret positions as well as grapheme
  myles: I imagine this'll be extended to other segmenters in the
         future, so keep that in mind.
  koji: Yeah, looking for opinions on that.
  koji: Currently the proposal is to add an attribute, and if you want
        to add different segmenters, maybe make it a function?
  koji: Or add other attributes that segment differently.
  myles: Will need time to think about it.

  <myles> koji: I really like how this doesn’t expose the concept of a
  <koji> myles: thank you!
  <krit> koji eae myles https://github.com/w3c/svgwg/issues/537 the
         issue of glyph counting in SVG by unicode code points _or_
         grapheme clusters.
  <eae> krit thanks!
  <myles> krit: it definitely shouldn’t be code points, that is
          certainly wrong
  <myles> krit: basically nothing in the web platform should be based
          on code points
  <krit> we agree in the SVG WG. SVG 1.1 uses it, it is implemented
         this way in at least one browser implementation and is
         reliably - always. It does not produce useful results in many
         cases though.

CSS Parser
  Scribe: iank

WICG draft

  <gregwhitworth> to be clear - this is the draft we're discussing -
                  not the one in houdini: https://wicg.github.io/CSS-Parser-API/
  glazou: Thanks for greg for starting document.
  glazou: Like to compare contents to one liner of abstract.
  glazou: "An API exposing the CSS parser more directly, for parsing
          arbitrary CSS-like languages into a mildly typed
  glazou: Can you build a pre-processor like SASS.
  glazou: There is extra things needed like the additional SASS syntax.
  glazou: I think we need to dive down into the lexer level.
  TabAtkins: That is indeed was is exposed.
  glazou: what about things inside {} inside a block?
  TabAtkins: No inside a block 2 choices, 1) @rules and decls, 2)
             @rules and other rules.
  TabAtkins: You need to tell the parser which one of those two modes.

  glazou: If you want to do something CSS-like.
  glazou: At this point we cannot, need to open this up a little more.
  glazou: A little too high level.
  TabAtkins: Want to expose the full token stream.
  glazou: If we do that implementing a SASS engine, will be doable,
          and much faster than the CSS pre-processors.
  glazou: We'll be able to ship SASS sheets to the end user, will be
          really useful.
  TabAtkins: Yes - faster and more correct.
  glazou: I still think exposing the lexer could be extremely useful.
  TabAtkins: Sure - can you file an issue?
  glazou: The lexers are quite difficult, are you open to such a
  TabAtkins: That's basically the css-syntax spec, which many browsers
             have reimplemented on top of.
  TabAtkins: Blink's parser is already close, as well as FF - I
  TabAtkins: At least servo's is.
  emilio: We use the same, so yes.
  <iank> Should be able to expose the token stream....
  Rossen: We are also improving ours to matching blink/servo's impl.
  glazou: Are you open do exposing.
  Rossen: Yes once we've moved closer.
  gregwhitworth: Let's get a v1,v0 of access to the parser, I need to
                 parse out the color, output is the typedom. v2 is the
                 lexer issue.
  gregwhitworth: I know there is a desire for custom @rules, you might
                not need that once we've get the lexer access.
  TabAtkins: TypedOM api first, then parser... (describes
  glazou: I'll have a little more time to work.
  gregwhitworth: You are referring to the WICG version of this?
  glazou: ...yes.
  Rossen: Ok sounds great, lets move on.

  Scribe: TabAtkins

Status Update

  majidvp: Wanna give an update on AnimationWorklet, I missed last f2f
  majidvp: We've made good progress on Chrome impl
  majidvp: Then a new interesting topic about inputs besides scroll to
           feed into, lots of interesting use-cases.
  majidvp: So this is AW background.
  majidvp: Timelines are new, they feed time into the animation, at
           the end you get a style update
  majidvp: WorkletAnimation is just adding a callback hook in the
  majidvp: So this lets you run script and have state, but you don't
           have to run on main thread.
  majidvp: We run on different thread, so we can keep up with native
  majidvp: No jank!
  majidvp: Last time we had a resolution on AW it was to migrate the
           repo, this has been done + all issues.

  majidvp: Another major change was to differentiate between stateful
           and stateless animations.
  majidvp: Stateless animations can pump frames in parallel.
  majidvp: So spec has been updated to handle that.
  majidvp: There was a destroy() to pull state out of an animation;
           now we have two classes, and Stateful animations are more
  majidvp: Spec's been updated, impl is lagging so far, will catch up.

  majidvp: Some demos...
  majidvp: Animation here is *very* simple, just pushing currentTime
           into localTime for the animation.
  majidvp: Most demos I showed are like this.
  majidvp: We're able to do this for both composited and main-thread
           scrolling with no problem. Getting less and less bugs filed
           by Surma, which tells me it's pretty stable. ^_^

  majidvp: Another update - initially we only updated the "fast-path"
           properties, but WG didn't want that to be special.
  majidvp: We have implemented that!
  majidvp: This demo was previously transform, but now it touches
  majidvp: So it matches what you can do with WebAnimation
  majidvp: Similarly we animate the fast-path properties on fast-path;
           slow ones happen on main thread.
  majidvp: Another feature we took from WA2 is GroupEffect.
  majidvp: We've exposed that inside of AW.
  majidvp: Lets you animate multiple elements together.
  majidvp: Doing an Origin Trial in Chrome71, to gather dev feedback
           on perf and ergonomics.
  majidvp: Questions?
  majidvp: A few more demos...

Generalized Time Values and Timelines
  github: https://github.com/w3c/csswg-drafts/issues/2493#issuecomment-422109535

  majidvp: So new proposal.
  majidvp: Early on we looked at things people needed for scroll, but
           we early knew we needed to solve the more general use-case
           of interactive animated effects.
  majidvp: Mobile or touch-friendly interface, you see drag-and-drop,
           scaling/zooming/transforming etc.
  majidvp: Touch-based interactions.
  majidvp: Right now only way to do this is with touch-event handlers
           on main thread and use rAF().
  majidvp: These are not time-based animations, just input-driven.
  majidvp: Scroll-linked animations are a subset of these.
  majidvp: Scroll is easier; it's 1d mostly.
  majidvp: So you can map it to time, easy. And can use with WebAnim.
  majidvp: Touch and gestures are more complex: not
           single-dimensional, and are stateful.
  majidvp: Almost impossible to do this declaratively in a way that's
  majidvp: This is why people use the manual scripting approach on the
           main thread.
  majidvp: What we want to do is use AW to handle these use-cases as
  majidvp: If we can pull this off, we can incentivize devs to move
           some critical UI rendering work off of main thread.
  majidvp: It also lets browsers know it's an input-drive animation,
           so they can prioritize properly.
  majidvp: We've all seen janky touch-based effects; it's very hard to
           pull off on the web.
  majidvp: So if AW can handle these, it should help a lot with the
  majidvp: And will have the same basic play/pause api we have for
           regular time-based animations.
  majidvp: So a lot of goodness here.
  majidvp: This is basically what we're going for scroll, but
  majidvp: Like to start with pointer inputs, you can do a lot of
           interesting things there.

  majidvp: So why AW as a primitive?
  majidvp: First it already runs off-main-thread. Designed so it can
           be stateful.
  majidvp: And because it's JS you can express complex things that
           would be very hard to do declaratively.
  majidvp: Main thing missing is having input other than scroll.
  majidvp: We've been thinking about this for a little while.
           Initially thought we could fit everything into timelines;
           maybe multiple timelines for animations.
  majidvp: I chatted with Brian and Antoine last time, and realized
           model doesn't work well. Multi-timeline doesn't fit well
           with animations, too complex.
  majidvp: I think our new approach is reasonable.
  majidvp: AniamtionInput is separate from AnimationTimeline
  majidvp: But touch animations can have multiple effects:
           pointer-input, scroll-input, etc.
  majidvp: More complex scroll too - 2d scrolling, maybe active scroll
           or momentum. Maybe know that finger is down, even if not
           actively moving.
  majidvp: So this keeps timeline simple, focused on that use-case,
           and then have the wider thing.
  majidvp: I want to build this model so later if there's a gesture
           exposed, maybe we can get access to those.
  majidvp: Also want to make sure that you can listen/unlisten to a
           given input.
  majidvp: Passively monitor something without it actually ticking you.

  majidvp: Conceptual diagram:
  majidvp: [on screen]
  majidvp: This is the high-level IDL: [on screen]
  majidvp: Example of PointerAnimationInput [on screen]
  majidvp: Proposal has a lot more details.
  majidvp: Here's an example of a more sophisticated scroll...
  majidvp: I like to dream, here's how a gesture might look.
  majidvp: [example code]
  majidvp: [example of mixing scroll+time]
  majidvp: Hidey-bar, for some portion you want to be linked to
           scroll, when user lifts their finger you want to complete
           animation based on time.
  majidvp: Set up two inputs...
  majidvp: Set actual timeline to empty, then provide the two inputs
  majidvp: This misses its timeline, even tho it takes a scroll as an
           input, so we can control precisely when scrolling will tick
           the animation. If provided as a timeline input, it would
           always tick no matter what.
  majidvp: So this is a high-level example, showing general direction.

  majidvp: So you might ask why not just use events?
  majidvp: For one, bubbling happens. We don't want to necessarily
           expose DOM via bubbling.
  majidvp: For two, pull vs push. We think pulling makes more sense.
  majidvp: One possibility - instead of this value concept, maybe have
           a list of events that occurred since last frame.
  majidvp: Maybe use event interface without actually inheriting from

  Rossen: So what are next steps?
  Rossen: Continue in WICG?
  majidvp: No, it's in Houdini already, we'll continue to work there.
  majidvp: I'm working with Brian now.
  majidvp: ScrollTimeline, WA, and AW have interdependencies.
  majidvp: Last time Simon told me to make the extension point clear
           in WA spec. We're currently patching WA.
  majidvp: So working with Brian, joined their monthly sync, to bring
           these concepts to that event.
  majidvp: Try to make progress on defining these.
  majidvp: And having this patchwork be properly introduced and
           integrated into WA.
  majidvp: Also hoping that ScrollTimeline will push forward at same
  majidvp: And hoping to get dev feedback on ergonomics.
  majidvp: That's on spec side.

  majidvp: For new things, I want to figure out this proposal.
  majidvp: I think there's a lot of value.
  majidvp: For moving interactive content off of main thread.
  majidvp: I think our experiment with getting scroll to work from
           animationWorklet has been informative.
  majidvp: Want to push it so we're in good place.
  majidvp: One last thing - people are thinking about introducing some
           form of input to Workers, so I'm gonna try to see if
           there's a general solution here for all these use-cases.
  majidvp: I'll try to implement in the polyfill first.
  majidvp: You can do all this with PointerEvents today.

Properties and Values API
  Scribe: gregwhitworth

Declarative property registration
  github: https://github.com/w3c/css-houdini-drafts/issues/137

  TabAtkins: Early on in this process, when we setting up the props
             vals API
  TabAtkins: we would want a more declarative version
  TabAtkins: you just want to animate a custom prop
  TabAtkins: you don't want JS
  TabAtkins: we delayed doing that
  TabAtkins: At this point, we're pretty stable with L1
  TabAtkins: That interface is working well

  TabAtkins: Time to revive this declarative proposal
  TabAtkins: Have some way to store it in your stylesheet
  TabAtkins: The shape of it is more or less what Amelia says in the
  TabAtkins: There are some minor changes for registration due to CSS
  TabAtkins: other than that - this appears to be relatively straight
  TabAtkins: There are some potential issues with timing
  TabAtkins: like a late registration not causing reparsing
  TabAtkins: It's easier in a declarative scenario - no need to go
  TabAtkins: overall I think it's reasonable and I want to deliver on
             providing declarative APIs
  SimonSapin: Just now you talked about early and late stylesheets, if
              the document contains 4 stylesheets does order matter?
  TabAtkins: If you have 4 registrations then the order matters
  fremy: It's fine
  fremy: I'm super excited

  futhark: Just a concern about timing and at-rules when they start
  futhark: You delay them, the latest would apply but what if one
           browser applies after two stylesheets but there's a new one
           coming in?
  TabAtkins: That's fine - you'll only be able to see it if you look
             at the GCS in between those loading docs
  futhark: So you would normally register the first time you collect
           the rules, the second time you would just ignore it?
  TabAtkins: Document order is normally how it works? but good question
  futhark: That's a little bit different how the JS version works
  TabAtkins: Yes because JS can be definitive
  heycam: What happens if you do both JS and declarative?
  TabAtkins: The script should just win IMO
  TabAtkins: That seems much more likely to have timing issues
  fremy: Yeah
  TabAtkins: Plus then - it saves the problem of dealing with multiple
             JS registrations

  heycam: Not sure what the current status is of @font-face or
          @counter-styles in shadow styles
  TabAtkins: As far as I remember, registering a property is global,
             so it will apply to all shadows as well - I THINK
  TabAtkins: I wouldn't want it to be different. @font-face and what
             not are scoped to their own root so it's different
  heycam: It's a bit unclear what "later in the document" means
  TabAtkins: It shouldn't be - you have a flat tree ordering
  TabAtkins: style rules that don't show up in the flat tree don't get
  emilio: They do apply
  emilio: if you have an unslotted style tree then it does apply
  TabAtkins: This is more complicated
  TabAtkins: [repeats question]
  TabAtkins: but if you have unslotted styles still applying, then the
             same question applies
  emilio: You use DOM tree order inside of the style root
  emilio: I'm not sure why you can't use DOM tree order based on Host
          for example, that's what I think is most reasonable
  TabAtkins: It sounds complicated - maybe we don't allow this in
             scoped stylesheets
  TabAtkins: You already have a shadow root, so you're already in JS

  gregwhitworth: Is there a plan for declarative creation of shadow
  TabAtkins: I've pushed for it - but so far no
  TabAtkins: We'll deal with it then I guess
  TabAtkins: Do people want to proceed with this?
  heycam: I think it's worth adding somewhere
  <bkardell> +1
  Rossen: There's enough excitement
  Rossen: let's pursue it somewhere, properties and values API?
  Proposed Resolution: Add it to the Properties and Values API
  Rossen: Objections?

  RESOLVED: Add declarative registration into properties and values API

CSS Animation Worklet

  majidvp: I would like to get a resolution for FPWD of animation
  majidvp: People wanted time to review it - we've worked on it... so
  Proposed Resolution: Publish a FPWD of Animation Worklet
  Rossen: Have people had enough time to look at it?
  Rossen: If not we can always resolve on this at our next confcall
  Rossen: Not hearing any objections - I'm taking silence as they're
          ok with it?
  Rossen: Any objections?
  dbaron: I'm fine with it going, I see a Google search leads to the
          WICG draft but it needs to go to the Houdini Draft
  astearns: In other cases it just redirects
  Rossen: Any objections?

  RESOLVED: Make Animation Worklet API as FPWD
Received on Wednesday, 26 December 2018 22:52:53 UTC

This archive was generated by hypermail 2.4.0 : Friday, 17 January 2020 19:53:28 UTC