- From: Dael Jackson <daelcss@gmail.com>
- Date: Sun, 21 Feb 2016 15:04:19 -0500
- To: public-houdini@w3.org
- Cc: www-style@w3.org
=======================================================
These are the official Houdini Task Force minutes.
Unless you're correcting the minutes,
please respond by starting a new thread
with an appropriate subject line.
=======================================================
Font Metrics
------------
- Typographic layout may be added to the layout API and this spec
will only contain metrics from the font and the style
attributes. No matter the end decision, the spec will start at
the smallest subset solving an important use case and grow
from there.
- The spec as written only provided the dominant baseline, but
there were several different arguments that the first version
of the spec should be able to provide more information,
possibly the entire line grid. This would also require a
method for the browser to say it doesn't know the information
if a font hasn't provided that information.
- There were four major topics that need to be addressed in this
spec.
1) What font is used (on what? first inflow glyph?)
2) information that browsers use to align text on the page
should be exposed, including the baselines.
3) a low level API for raw (often wrong) font metrics.
4) font geometry detection / font metrics synthesis.
- SteveZ and astearns will work on the base line table
- eae and dbaron will look into exposing why font is used
- Inferred geometric data will be deferred.
Composited Scrolling and Animation
----------------------------------
- There are five main use cases that have been mentioned for
composited scrolling and animation:
1) parallax
2) scroll headers
3) video sync
4) linked scrollers
5) drag and drop
- The hope is that the group can decide together what are
important and if this is the right place to solve them.
- Google is currently working on implementing the approach listed
in the spec. Most other browsers didn't respond, but Apple
said they would likely follow a similar approach to Google's.
- There was interest from Apple in solving as much of the use
cases as possible using declarative script so that browsers
can optimize.
- This led to a wider conversation about how to ensure that
items were ready for the fast path or at least telling
authors if they had something that wouldn't go on the fast
path.
===== FULL MINUTES BELOW ========
Agenda: https://github.com/w3c/css-houdini-drafts/wiki/Sydney-F2F-January-2016
NOTE: The group split into two concurrent sessions, one for font
metrics, the other for composited scrolling and animation
Font Metrics
============
Scribe: gregwhitworth
astearns: When I originally brought up font metrics, I was
thinking more text measurement.
astearns: There are font metrics that we get from the fonts
themselves and style attributes,
astearns: then there are the typographic layout.
astearns: Maybe typographic layout may just get added to the
layout API.
astearns: The font metrics may be desired by someone for custom
position of the glyphs.
florian: If you have a nested, text you may need stretching of the
glyphs to wrap around the text.
florian: For example math specific fonts have info for this.
shane: That sounds like we need two different font APIs.
florian: The author of mathJax really has a desire for this
capability.
astearns: There are two topics and they are very, very separate.
astearns: Does anyone else have other topics?
florian: Let's throw use cases as well.
florian: For example, you would need the this info for first letter.
astearns: It really depends. But if you try to do things only with
text measurements, you end up doing layout to get the
measurements and do layout again to achieve what you want.
astearns: I think you always need the measurements, but you may
not need the glyphs.
dbaron: Do we want to expose different phases of the font process,
such as shaping?
astearns: We want to expose as little as possible and the smallest
subset that solves an important use case.
astearns: And keep adding to it, rather than exposing a large API.
dbaron: You may want a measurement where you pass it a string of
text.
dbaron: You may also want a measurement of the text with certain
styles.
astearns: That's a little different than I was thinking of.
astearns: There's also, just layout results.
shane: Something to keep in mind is that the layout is expensive.
eae: You could use measureText API to form canvas, you can get the
baseline or baselines in some cases.
astearns: You can, but it gets drawn into the canvas by different
browsers.
eae: Yes, it's very limited, but it's better than nothing,
shane: It's like 50x faster.
astearns: I want to avoid putting in divs just to find out layout
information.
astearns: I have two blocks that are laid out, and I want to align
the second box with the first one's baseline, you're
simply adjusting the info.
shane: I think you should do specific layout into containment
regions.
eae: Right, you can give it an offscreen area and run this stuff.
florian: I think it's cool and important to be able to trigger the
API without triggering the rest of the pipeline if you
don't care about it.
astearns: I have done an API before, the current layout API shows
just a baseline, there can be a glyph, a run, and each
has a baseline and they can differ.
astearns: But it has been carefully defined what you're talking
about.
astearns: The most popular one is the dominant baseline of the
first character of the first run, and it was
tremendously useful.
astearns: Making it available at each step of the tree makes it so
that you don't care about the glyph.
astearns: Saying that a block has a dominant baseline glosses over
a lot of complexity.
florian: Yes. It's wrong, but it's useful.
shane: Well it's not wrong.
florian: Is it useful?
stevez: Is that enough to do line-grid.
astearns: No.
stevez: That's what I thought.
eae: You can also imagine segmenting on the different baselines.
florian: You still need to know all of the baselines.
florian: The line grid itself needs to have them all so that you
can align to it.
stevez: We could avoid needing to document to where we ask the
interface for specific ones and if it doesn't know it will
say it doesn't.
astearns: Once you get the content rendered, the browser has
determined the dominant baseline.
stevez: One area where you need a baseline and top line are in
international languages for top cap.
shane: These lines are in the font?
florian: No.
florian: Some.
stevez: You're lucky to get the alphabetic if anything.
shane: I'm just curious if we'll ever be able to lift this up to
JS, it may just need to be in the font engine.
florian: That's my issue with this discussion, while I don't want
to lift this to JS, but when I look at the use cases we
need this information.
shane: I think it's ok that if something isn't there in the font,
it's ok for us to say we don't have the information.
stevez: When fantasai, dauwhe, and I discussed the for CSS inline
the best to generate values that are missing from the font
information.
florian: Tying it to your point *points at shane* it should be up
to the UAs to be able to put their hands up.
shane: I think you should be talking to the font engines.
iank: My only perspective is that if this doesn't force layout
into a corner I'm OK with it.
shane: We should have an API that provides a null response if we
don't know, but when you want a response we provide the
dominant baseline
astearns: There could be a fallback.
Bert: Let's say you want to know the cap height, does it say
something?
florian: No, you're forcing the browser to lie.
stevez: What if the browser is using the cap height?
florian: Then the browser should tell us that.
stevez: Some of the Adobe products create font information for
cross platform interoperability.
[gregwhitworth says something about font fallback]
astearns: And that's a different need where we need to know about
which fonts are used on a given range.
stevez: Could we live with just getting a fixed number of things
in a baseline table (alphabetic, ideographic, etc)?
astearns: But those are things that the browser DOES synthesize.
florian: Do you envision asking the element foo for it's baseline
and it gives you if it knows it.
stevez: Different. If you implement L1, you get the basic ones.
shane: Your adobe tool, does it run in real time or does it cache
it?
stevez: I don't know really, I think it may be both,
astearns: Unfortunately there are a lot of typographic engines in
adobe products.
<eae> In summary, my point was that if the browser needs to be
able synthesize a baseline for other reasons that
synthesized baseline should be provided as a part of the API.
shane: We should possibly try and standardize the heuristics that
Adobe uses in these tools if possible.
shane: Maybe we provide devs with the opportunity to choose which
path they want, cross platform or matching the system.
eae: But you have to provide all of the paths to correctly match.
stevez: This will be very hard on Apple as they don't always
provide the correct info.
astearns: I think we can simplify even further for V1, we just
provide that first dominant baseline and provide the
table in future versions
stevez: I wanted to start off with the table so that people start
thinking in that direction rather than coming in with a
new interface.
astearns: Yeah, ok.
eae: *shakes head up and down* I think that's a really good idea.
astearns: You would be able to determine which baseline is the
dominant one?
florian: Yes, I think you should.
stevez: Ideographic characters are positioned in one of two ways,
either at the bottom of the em box or at the center.
stevez: Those are both used, so if you give the em box position in
the table then that would be sufficient as well.
florian: Then with math finding the center if you have the top and
bottom it would be easy.
koji: Vertical is always center, horizontal is either bottom or
center.
stevez: You're going to miss cap height on any font that isn't
western.
astearns: You'll have ascent and descent but they aren't helpful.
florain: And they'll all be buggy I suppose.
astearns: Yes.
stevez: Cap height is not reliably available.
<liam> [ cap height ascender/descender height and x-height where
available are incredibly useful]
astearns: Right now the text measurements are hanging off the
fragments, I suggest rolling this up to the element and
it will be the first inflow fragment.
stevez: But the element may not have gone through layout.
shane: Yes we have hit issues with conflating the left of the line
and we should ensure that this doesn't happen.
astearns: Yes we need to determine if we should allow custom
layout stuff to potentially allow layout to be invoked
since ergonomically it's what devs are used to.
shane: There are strong opinions within the Chrome team against
doing that.
shane: I think going down that path is bad for the web
stevez: So then we agree it's on the right of the line (of
Rossen's process model design).
stevez: When he says the first inflow run, the drop cap will be
wrong since it's out of flow.
stevez: If the baseline is taken from the first inflow glyph, then
floats and drop caps are not inflow.
shane: So it needs to be in text, not inflow.
stevez: What's text?
eae: It's special.
stevez: We're changing the interface to have alphabetic, top,
bottom and parameters for which is dominant.
florian: Can it happen that the dominant baseline is not one of
these? It should be yes.
stevez: For practical purposes, is no.
stevez: Good point,
stevez: you could define ideographic center.
eae: Or we could just provide the position of the dominant
baseline rather than the name.
stevez: That allows us to add additional baselines without
changing the interface.
heycam: Are we talking about glyph data?
astearns: The input of the metrics or the result of layout?
heycam: The actual font glyph info.
Scribe: Florian
shane: Do the type of raw metrics diverge between font types?
all: Yes.
dbaron: Font by font even.
SteveZ: The tools people use sometimes plug in values without
justification, and you may be better off when the tools
remained silent.
dbaron: All fonts form a foundry may have the same data,
regardless of the font.
Bert: Can I find out where the ink is?
SteveZ, astearns: There is bounding box data, it sometimes is wrong.
Florian: How comes anything works if all font info is wrong?
dbaron: Hard coded font specific heuristics.
astearns: Several libraries do manage to extra useful information
and do things with them, so we are not wasting our time.
astearns: We should expose the raw font data, knowing it has
issues, and let JS deal with it,
astearns: but we have to expose the actual font being used.
<liam> [when the information gets used, over time, some of the
errors get corrected, people fix their fonts]
shane: There are 3 categories of things we can do
shane: 1) What font is used (on what? first inflow glyph?)
eae: What if you use multiple fonts for a single glyph?
eae: We have APIs exposing the list of fonts used in a list.
heycam: How do you expose this?
astearns: Some object that you can use to get metrics from.
dbaron: An object that represents a face.
gregwhitworth: How do you untangle things when all fonts are used
together?
shane: Don't dive into the details to quickly.
shane: 2) information that browsers use to align text on the page
should be exposed, including the baselines.
shane: 3) a low level API for raw (often wrong) font metrics.
shane: 4) font geometry detection / font metrics synthesis.
shane: We should identify a leader per area.
SteveZ: We need to get something written down at the level just
described.
[SteveZ + astearns will do base line table]
[eae + dbaron will look into exposing why font is used]
Florian: Do we need to work on raw font metrics in level 1?
shane: While exposing which font is used, we should also include a
way to get the raw font file, so that people who want to
parse it themselves can get to the data easily.
<liam> ["get the data easily" easier said than done e.g. for a
graphite font!]
astearns: There is a risk of fingerprinting, but the cat is out of
the box already.
dbaron: The only way to close this is for browsers to ship their
own fonts and not use OS fonts.
dbaron: We shouldn't expose raw font files for local fonts, only
for webfonts.
shane: For inferred geometric data, we should defer.
all: Agree.
eae: There's a 5th area: line break opportunities?
florian: That's useful, but separate topic?
dbaron, astearns: I agree, it's a separate topic.
shane: And I'll write the high level explainer
Composited Scrolling and Animation
==================================
Scribe: vollick
rbyers: We're hoping to avoid API debates and focus on use cases
and discussing other approaches.
rbyers: We'd also like to know about constraints of other
implementations.
rbyers: As well as knowing what data would be helpful for us to
gather.
rbyers: Ian and I have been scrambling to get together a list of
use cases in the repo. Would be great to collaborate on
this list.
rbyers: Anyone else have suggestions on how to make this time
productive?
rbyers: We've been thinking about this as two separate things. 1)
compositing and asynchrony. 2) scrolling extensibility.
rbyers: We'll start with how to plug into compositing and access
to that thread.
rbyers: surma was going to present a demo that he's built to start.
surma: This is slightly distorted, but you'll get the gist.
surma: This is a test build that has a version of CompositorWorker.
surma: I've hooked into scroll position to transition a header.
surma: This permits a very high frame rate effect.
surma: The response on twitter has been enthusiastic.
surma: This is probably the proposal that would get quickest
adoption.
surma: I will try to share the demo as soon as I can.
rossen: Can you explain how this was built?
surma: Currently I could only use pixels for transform presently,
but this is bald html/css with JS that hooks to scroll
position.
jet: Is it the fact that you're just talking in pixels what makes
it fast?
surma: I don't think so, it's that it happens on the thread. Could
probably handle other units in the future.
<surma> Here’s a video of what I just demo’d:
https://www.youtube.com/watch?v=EUlIxr8mk7s
Scribe: rbyers
vollick: Another demo: it's a toy,
vollick: but it's a physics simulation happening in the compositor
worker.
<rbyers> https://github.com/w3c/css-houdini-drafts/blob/master/composited-scrolling-and-animation/UseCases.md
vollick: Jank that's happening on the main thread isn't affecting
the physics sim / animation.
vollick: We've been working on an implementation of compositor
worker in the hopes that we can ship position:sticky and
snap points in terms of it.
vollick: Largely so we can learn about the performance
implications. We'll share that once we've got it.
vollick: The initial compositor worker code is almost entirely
landed in chromium trunk now.
vollick: Let's walk through example use cases for composited
scrolling and animation.
vollick: 1) parallax
vollick: Maybe not very interesting since you can do it with 3d
perspective.
dino: Really? seems pretty interesting - may want to do different
sorts of math
smfr: I'd say compositor worker is the natural way to do parallax.
dino: Seems like one of the most important use cases
vollick: 2) scroll headers
vollick: 3) video sync
vollick: This is an example brought up by dino - video position
coupled to scroll position
vollick: 4) linked scrollers
vollick: 5) drag and drop
vollick: Want elements that stick to your finger, regardless of
what's happening on the page.
rbyers: To step back for a second, these use cases are not just
for CW - we'd like to collectively figure out which ones
are important and brainstorm other solutions.
rbyers: e.g. dino's "animation time bases" proposal solves some
of these.
dino: Actually, it doesn't "solve" them - just enables a couple
specific scenarios.
vollick: It covers other use cases - drawers, performant effects
libraries, gesture recognition, element location tracking
vollick: [scribe fails to keep up] other use cases
vollick: any objections
[rbyers and smfr agree that we can leave raw input to v2 of the
API since there's complexity to explain hit testing and other
things]
Scribe: Rossen
jet: Is the idea of compositor worker to be able to keep up with
the fast path only? if we have getComputedStyle we'd fall out?
vollick: This is certainly the hope
vollick: [showing examples of scrollmagic.io]
<vollick> https://github.com/w3c/css-houdini-drafts/blob/master/composited-scrolling-and-animation/Explainer.md
vollick: Some of the examples should already work.
rbyers: Should we move the input example to a different place?
[discussion of how exactly the input example works]
vollick: The inputs to the callback are traits and not properties.
dino: If a compositor worker is killed, what happens?
vollick: It won't work, assumption is that it will not.
vollick: I'm hoping that we'll be able to use a worklet for this.
rbyers: An important question/design goal is to be
suspendable/resumable
smfr: How could you do collision detection?
vollick: It's tricky.
smfr: From a given compositor worker can you access multiple
proxies?
vollick: Yes.
dino: You can be getting more post messages?
rbyers: Yes but it'll depend on what we use.
vollick: Another idea - being able to notify the user they're
behind.
dino: raf and worker runs after the main thread?
vollick: We do best effort in the same frame.
dino: Changing a background color based on manipulation?
rbyers: Yes, this is a good used case we should consider.
rbyers: We don't want this to be the new feature that introduces
even more jank.
vollick: That's about it.
dino: about what exactly?
Scribe: esprehn
Rossen: What do other vendors think?
dino: Google is moving ahead, what do other browser vendors feel
about this?
dino: We might standardize something very close to Google because
we don't have a prototype yet.
rbyers: I'd like to get to the point of figuring out what's next,
ideally next time we have it behind a flag so developers
can use an API key with the experiment framework to try
this out.
rbyers: At that time I think we want to really make sure we get a
solid spec that's not just impl details.
philipwalton: Is there a declarative version of this?
rbyers: dino's proposal is as close
rbyers: Actually lots of CSS features are close to this like snap
points, etc. could spend 10 years making all the various
declarative things.
philipwalton: What about cases like the width of the progress bar
at the top that's based on the scroll position of
something on the page?
rbyers: No spec for that yet, there's animation time base which is
in the same space.
Rossen: You're asking if we can make the scroll offset into a
value type in CSS.
philipwalton: Yes, in CSS set a function of the scroll offset of
some element; width: scroll(#id)
dino: Yeah we have use cases, not actually the scroll top, but
it's a special calc like scroll height - some other things,
need to invent more new features for this.
dino: For simple parallax there's probably a way to create a
declarative syntax.
smfr: Other way is like web animations where you setup a thing on
the main thread and run in async.
rbyers: Setup a graph data structure.
smfr: Don't really need script to just do .8 * scrollTop.
rbyers: I don't want us to be afraid of running a tiny bit of
script every frame.
smfr: Should be afraid of adding scripting hooks where authors can
make mistakes.
vollick: My hope is we can come up with a shape that alleviates
some of these mistakes.
smfr: No, difference is script vs declarative is the UA can
optimize the declarative approach,
smfr: script based approach you can't optimize (the algorithm).
smfr: Only mitigation is put it back on the main thread when it's
too slow.
smfr: I'm not totally against this, I would just prefer a
declarative approach.
vollick: I know you haven't had time yet, but it'd be great if we
could look at the use cases with the declarative approach.
rbyers: I have a hard time imagining how this would work for all
the use cases with the declarative graph language.
rbyers: Like Google did where cards spread out as you scroll.
dino: Yeah, Apple does that for messages.
rbyers: If we can come up with how to redo this with some
declarative language I'm okay, but it seems hard.
rbyers: It seems developers may not have enough tools.
smfr: But developers would probably tell us when things were
missing.
dino: It is limiting, but maybe not as much as you think if we get
enough cases.
vollick: It would be nice if there wasn't a foot gun to avoid
developers fall into the slow path such as when animating
the width.
ojan: Yeah, it's bad to only have the declarative approach because
you end up trying to solve all use cases.
ojan: For example flex box has n^2 trying to solve all the cases.
smfr: You don't need to solve all cases.
dino: Yeah.
ojan: That's really hard.
ojan: I would have both in an ideal world, encourage declarative
approach since we can optimize, but have fallback.
rbyers: It's the same argument for custom layout.
rbyers: Developers will prefer to use built in CSS things, and
build things that are popular in custom layout libs back
into the browser
rbyers: but custom layout allows innovation.
ojan: I'm okay doing declarative but I'm worried if we only do that.
ojan: It's a mixed blessing.
ojan: You can find cases where it's terrible and cases where it's
great.
rbyers: What if we said you can't position objects with JS, you
must use CSS?
rbyers: I don't think we would want to do that.
ojan: I'm curious about dino's question about what is Microsoft
and Mozilla sentiment.
Rossen: What thing?
rbyers: The concept of giving developers a thing that's typically
synced with scrolling.
Rossen: It's pretty crazy, on our side we're trying to do all
kinds of compositor optimizations using the lower level
parts of the system which are fairly restrictive, even for
us, to know at any given time where things are.
Rossen: And if at any given time we have to go look at script to
find what to do that's going to throw away a lot of
optimizations.
Rossen: I'm not sure it's even a question of declarative vs
imperative.
rbyers: You mean both fight with the lower level optimizations?
Rossen: Yes, so far we've found most ways to work around the
declarative things since ahead of time you can categorize
and figure out what to do later.
Rossen: If you don't know until you get on the right side you
can't optimize.
Rossen: Even if all your script callback does is simple like * 2.
smfr: You can limit what's allowed in the declarative approach to
allow keep being efficient.
vollick: It's the same as having a limited set of animatable
properties in core animation.
jet: Mozilla is working hard to make things that fall off the fast
path fast, like animating left.
jet: They would not want to be painted into a corner, saying
things like you can't put left on the compositor.
vollick: This design allows UAs to expose new properties.
vollick: For example you can ask if a proxy supports a property.
rbyers: We wouldn't support properties we can't make fast.
rbyers: So the concern that servo still stands, since devs won't
use a feature which is slow/doesn't work in all browsers
that isn't servo.
dbaron: I think I'm reasonably supportive of the general idea
here, but it's sort of one of the things where we're
freezing today's architecture into a standard more than
usual.
dbaron: Is this still useful in 10 years?
ojan: I share that concern.
dbaron: Is the compositor thread a thing in 10 years?
ojan: I agree, that's why we switch to worklets.
rbyers: The UA could also choose to do it on the main thread in
this design (the one with postMessage + compositor proxy).
vollick: We could run this on the same thread, in that world this
is a win because it limits what the code can do.
vollick: I think the primary issue is if something we put in this
subset is slow later, which seems rare, do you think so?
smfr: I agree.
ojan: It depends what your subset is.
dbaron: The case I can think of is inverted color drawing.
rbyers: Do we move to scroll customization?
dino: No, lets break now.
[break]
Received on Sunday, 21 February 2016 20:05:20 UTC