- From: Dael Jackson <daelcss@gmail.com>
- Date: Tue, 3 Mar 2015 14:20:11 -0500
- To: public-houdini@w3.org
Houdini-Related Presentations Continued
---------------------------------------
- After lunch, rbyers brought up allowing more controls for
scrolling, especially to allow extensible scrolling so that
things like chain scrolling and infinite scrolling can be
done and explained.
- bkardell brought up being able to explain and define isolation
which lead to Rossen leading a conversation about better
definitions of what breaking means and how to create more
break controls which has been of especial interest for
E-Publishers.
- astearns brought forward the creation of something that he's
calling "style mutation observers" to trigger an event when
a style changes.
- Lastly, Rossen brought up the box tree as something crucial to
define and decide how to expose in order to make almost any
of the previous topics possible. The group leaned toward
first creating a definition of the box tree in a static
state and then giving hooks into the box tree to make it
dynamic.
===== FULL MINUTES BELOW ======
Houdini-Related Presentations con't
===================================
Scrolling Extensibility - rbyers
--------------------------------
Scribe: shans
rbyers: I think scrolling extensibility is related to this area
too.
rbyers: I'll just paste in a couple of links.
<rbyers> https://docs.google.com/a/chromium.org/presentation/d/1P5LYe-jqC0mSFJoBDz8gfJZMDwj6aGeFYLx_AD6LHVU/edit#slide=id.g4119c536d_015
<rbyers> https://www.youtube.com/watch?v=L8aTuoQWI8A
<rbyers> https://docs.google.com/a/chromium.org/document/d/1VnvAqeWFG9JFZfgG5evBqrLGDZYRE5w6G5jEDORekPY/edit#heading=h.kd0gtwwz5bf9
rbyers: This is a talk I gave at BlinkOn. Summary: native apps
have lots of interesting scroll effects, we want to be
able to do this natively too.
rbyers: The most extreme example is pull-to-refresh.
rbyers: Working on a few primitives that might help.
rbyers: First thing is need some ability to synchronize with
scrolling.
rbyers: e.g. if transforming something around the page,
rbyers: scroll-blocks-on is what we've exposed in canary.
rbyers: Every browser already has some notion of what scroll
blocks on, it's good to expose that.
<rbyers> https://docs.google.com/a/chromium.org/document/d/1aOQRw76C0enLBd0mCG_-IM6bso7DxXwvqTiRWgNdTn8/edit
rbyers: This property gives you a way to control that.
rbyers: For example, stop waiting for touch events:
rbyers: Lots of potential concerns: We want to make sure we don't
provide a footgun.
rbyers: It's still very experimental.
rbyers: I don't want to do anything in isolation, hope we can talk
about in this group.
rbyers: Synchronizing with scrolling can be done two ways - either
by forcing everything to happen on the main thread, or via
extensible scrolling.
rbyers: Also, how do we explain things like scroll chaining?
rbyers: e.g. in pull-to-refresh, the spring needs to collapse
before anything else scrolls.
rbyers: We should be able to explain overflow scroll property and
build it in JS.
rbyers: I've got a proposal called scroll customization that
provides custom scroll and custom scroll-distribute
methods to change how scrolling is implemented.
rbyers: It's super prototype, but I'm hoping others will be
interested in talking about this topic.
rbyers: I'm happy to give demos etc. if people want to talk more.
glazou: The topic that triggered the original discussions was the
box tree. We haven't discussed that yet.
astearns: During lunch we talked about the best venue for this
discussion.
astearns: Maybe the Houdini list is better than the style list.
rbyers: I think that was a slightly different problem - threading -
but it's related. We probably don't want to talk about
threading here.
plinss: With scrolling, in the early days of the gecko engine we
had a choice of how to describe scrolling in terms of
internal models.
plinss: Internally had an anonymous box on the outside and another
on the inside.
plinss: But are you moving the layout of these boxes around, or is
it just something that happens at render time?
plinss: This is going to be an interesting one - there's lots of
cases that potentially fall on both sides.
plinss: This is going to be hard to standardize because of
differences in implementation but I think we should.
plinss: If something is overflow: visible and scrollable through
script, and other things wrap around it live, then that
needs to happen at layout.
rbyers: With the scroll customization primitives, if you first
want to build something like overflow: scroll, then with
the box model primitives that we're talking about should
be able to meet some of these without adding extra to the
platform.
ChrisL: From a user perspective, some things that look like
scrolling aren't. e.g. tumblr model - things are actually
being evicted from the top. But it looks like scrolling to
the user.
ChrisL: So decoupling gestures and presentation from back-end
implementation.
rbyers: We want to decouple the intent from what that intent
triggers. That's our thinking too.
rbyers: Infinite scrolling is a good example, but we can build it
on top of scrolling.
rbyers: But and example is what if I want to scroll through a
spinning wheel and come back to where I started. You can't
do that today.
rbyers: We should be able to implement a scroller that under the
hood is a 3D rotate.
ChrisL: We also can't do scrolling as navigation mechanism.
plinss: Infinite scroll is a really good example of why we need
the box tree. Sometimes infinite scrolling is just
capturing the gesture. Sometimes it's lazy loading.
plinss: We should be giving devs primitives to do these properly.
Should, for example, be able to say that there are 10k
items, 500px high, without them being explicitly in the
DOM.
plinss: You need to get to primitives that let these things happen
without breaking everything else.
Breaking/Isolation - bkardell and Rossen
----------------------------------------
murakami: Are those things related to page views included in the
agenda?
Rossen: I don't think there's a fixed agenda, but we should
definitely talk about that.
<bkardell> No one has discussed isolation.
bkardell: One thing that I think is worth talking about is
isolation.
bkardell: That's really dominated web-apps and www-style.
bkardell: What we have currently is almost impossible to manage in
certain scenarios.
bkardell: Developers really struggle with it. Worth discussing
what the missing primitive is and where it belongs.
<bkardell> https://briankardell.wordpress.com/2015/01/14/friendly-fire-the-fog-of-dom/
<astearns> pagination requirements from digipub group:
https://www.w3.org/dpub/IG/wiki/Pagination_Requirements
Rossen: One more topic: haven't heard yet about how we deal with
breaking, and what breaking means.
Rossen: Line breaks, box breaks, page breaks (without defining
pages).
Rossen: One of the ideas that I wanted to include as part of the
next level of fragmentation is the ability to have named
breaks.
Rossen: We can let people define their own types of breaking.
Rossen: e.g. I'm defining my own page element and want to know
when something inside has reached a breaking boundary,
Rossen: And then handle what happens.
Rossen: We currently have the names 'column', 'region', 'page' and
the platform deals with them,
Rossen: There's no reason why the host shouldn't be able to define
what the handling is.
Rossen: Our implementation is pretty much the same and we use it
for everything.
Rossen: It's the primitive we use for breaking layout between
fragments.
astearns: Can you describe a custom break type use case?
Rossen: Yes.
Rossen: We've been getting lots of feedback from applications that
do EPUB content presentation.
Rossen: It's really hard to start layout somewhere in the middle
unless you control the content entirely.
Rossen: e.g. if I try to load the bible on an 8" device, chances
are that there will be thousands of pages.
Rossen: If I want to resume reading in the middle, it's difficult
to know how to do that without doing layout.
Rossen: Having breaks and being able to do layout once for that
size and font set lets us resume without necessarily
having to chunk the content.
<ChrisL> break-book break-chapter break-verse
Rossen: I want named breaks because I'm sick of arguing about
pages.
Rossen: It should be whatever you want to call a page.
Rossen: If I can create a pagination context that understands
breaking 'foo', then I should be able to generate breaks
in 'foo' from content and handle them in the 'foo' engine.
bkardell: Would this have an impact on the URL?
bkardell: How would you load a page from the middle outward?
bkardell: Is the bible a single HTML page in your example?
Rossen: Yes.
bkardell: There's lots of ways the bible can be divided.
bkardell: At some level this could potentially effect URLs -
should it?
johanneswilm: As I understand it, the entire bible is one HTML
page. You load it, but first the developer has
already done a pagination run. So page 316 could
immediately jump to the right place in the content.
johanneswilm: So how would this work? Right now I'd count letters
per page and store it somewhere on the page.
johanneswilm: Then I'd sum the first 315 pages, see how many
letters, then jump to the exact point in the text.
johanneswilm: But how would you store these counts?
johanneswilm: Should you have to display once? Or should this
happen without ever having loaded?
<bkardell> But I can share a link to Genesis chapter 1 vs 3 on the
web - and that's important.
Rossen: Today we don't have a primitive that exposes the break
record.
Rossen: Usually it's a collection of all the elements and boxes
that have been broken, plus enough context to describe how
much content was consumed and how much is remaining.
Rossen: Without it being exposed you can't just resume generating
page.
bkardell: So I can share a link to Genesis chapter 3. But if the
URL doesn't change, how could you do that?
bkardell: It feels like this should be holistic.
Rossen: You should still be able to find the position of a named
element and resume layout from the nearest page break
before that element.
<johanneswilm> +1 for allowing user defined breaktypes. Check this
example of how it can currently be done:
http://fiduswriter.github.io/simplePagination.js/simplePagination.html
bkardell: But I can't share that link if it doesn't have an
affordance.
glazou: This isn't about sharing. Concrete example - you have a
document in your browser, you've scrolled, you close the
tab, then you reopen it. You're in the same position.
Everything had to be re-laid-out because that's the only
way it can work right now. Rossen wants to optimize to
layout from nearest break. He doesn't want to change
anything.
bkardell: If that were a possibility I imagine we'd use it a lot
more. I'm wondering if that can also be backed out so
that there are URLs to each page.
ChrisL: You do this with identifiers in the HTML, then declare
there are page breaks.
<bkardell> Thank you, that's what I was asking for.
ChrisL: You don't share the page (different devices allocate
differently to pages).
ChrisL: Instead, it's so that if you go back to the device then
you get the pages faster.
ChrisL: But someone else might have different pagination.
glazou: Why do you need user defined breakage?
Rossen: You can only break on column, region, or page right now.
From an application's point of view, if you're creating
your own paginator and view, then I want to declare which
breaks I honor. (e.g. I handle article boundaries but
don't paginate the articles).
Rossen: So with user defined page breaks I can define what
'article begin' looks like and that I can break on article.
glazou: It seems like regions are a solution here?
Rossen: That's what we thought,
Rossen: But some cases are a little bit hard.
<dbaron> What does this have to do with the agenda?
<bkardell> dbaron: It just has to do with whether the idea loses
an important quality of the web if you open that door.
<bkardell> dbaron: I think it sounds like it doesn't necessarily,
so it's not important.
Style Mutation Observer - astearns
----------------------------------
Scribe: rbyers
astearns: I want to discuss "style mutation observers"
astearns: We need to know when a style property changes.
ChrisL: Any selector would have an observer which triggers an
event when what the selector matches changes?
astearns: Yes.
<bkardell> jquery.live is essentially this
<bkardell> http://api.jquery.com/delegate/
<bkardell> difference with jquery is it is for events
glazou: Selector matching observer or a property computed value
changing observer?
astearns: That's a good question.
astearns: New property - want to know what selectors it's being
defined in.
astearns: When property value changes.
glazou: OK, so you need both.
ChrisL: Like DOM mutation, if nothing is watching there's no
downside, right?
ChrisL: Not doing anything doesn't cause any downside.
glazou: True, except you still have to check the number of
observers.
dino: Why is your API just about selectors rather than detecting,
for example computed style changes?
dino: I assumed "style mutation" could detect when style on an
element had changed.
astearns: I'm not sure what all needs to be tracked. What
selectors match seems to be the most important thing.
astearns: It will happen much more often than the computed style
changing,
astearns: But maybe it's better to track the computed style
changes per element.
<bkardell> Maybe you'd set it up like mutation observers - watch a
selector and optionally only when the style changes
glazou: You probably really want to know about the rule, not the
selector,
glazou: And you want to climb up the cascade.
glazou: We suggested we add it to the OM a long, long time ago. It
existed in Gecko. I suggest we add it.
roc: If you're polyfilling a CSS property, what dino said is right.
When does a computed value change (including element
added/removed from document),
roc: You want that notification for all elements.
roc: You don't need to know anything about selectors.
shans: I think webkit had a similar API for walking up the cascade.
Both WebKit and Mozilla had to remove it for security
concerns.
shans: I'm not saying we shouldn't do it, but need to build
awareness.
shans: It was an API that gave you a set of style rules that
matched a given element.
<bkardell> I think you can build this on mutation observers today
with just .filter and .matches right? where is the
security issue?
roc: You don't need that.
roc: That's what people polyfilling CSS properties need.
roc: Editors need other things.
astearns: Could this just be built on top of DOM mutation
observers? Triggers on computed style?
roc: No, that's not how you'd do it.
shans: I used this once to polyfill CSS events.
roc: That's different than CSS properties.
shans: Focusing on narrow use cases like user custom properties
then you don't need it, but other cases do.
glazou: I have a concrete example.
glazou: Editor, style property span on it.
glazou: Works whatever the style applied to the document - may be
complex stylesheet.
glazou: Will try to find the deepest rule to modify.
glazou: All the machinery is hidden to the user. User just says "I
want to set the font white here".
<bkardell> glazou devtools must essentially have this, right?
dino: Why does that need to be web-facing?
dino: The devtools API has this.
dino: Could be standardized outside this group.
glazou: IMHO it's extensibility of the style engine - so fits here.
<bkardell> glazou are you essentially saying that you should be
able to have an element and say "what rules am I
matching right now?"
dino: I need to understand what use cases we want to address. roc
and I are talking about very different use cases.
Box Tree - Rossen
-----------------
Rossen: Any other topics?
Rossen: One topic not put forward explicitly is the box tree.
Rossen: This is the one topic that lured me into all this.
Rossen: Most of the topics discussed today would require deciding
what the box tree is and what it's API surface should be,
then tacking on additional behavior.
Rossen: But the parser and style side is just as important.
<bkardell> Parser doesn't require box tree as far as I can tell,
nor do custom pseudos?
ChrisL: It's like CSS 2.0.
ChrisL: Where we said there should be a clear separation between
the document tree and style tree,
ChrisL: But one vendor had a single tree and blocked this.
Rossen: Yes, it's time to change.
Rossen: The box tree sounds like a fairly major piece that's worth
discussing.
Rossen: I would be a good starting point.
Rossen: I want to hear ideas for how we should expose the box
tree, what APIs should look like, what hooks should we be
giving.
<bkardell> Isn't it necessary to concretely define the box tree in
an agreed upon way before you start talking about
exposing it?
roc: I think it's relatively easy to expose a read-only non-live
information about the box tree.
roc: We need to define what it is, but mainly an editorial issue.
roc: The exact details of getting browsers to match those specs
may be non-trivial,
roc: But exposing in a read-only way shouldn't be too hard for any
of us. It's needed for drawing borders correctly etc.
roc: It has to be a non-live API-
roc: A snapshot of the current state in time.
roc: Different engines don't have these boxes sitting around.
Tracking changes will be really painful.
ChrisL: You still want people to change the box model, just not
directly right?
roc: Providing mutation should be a separate problem.
roc: It's best to separate the boxes that are under the browser's
layout control from those under developer's layout control.
roc: Simplify things if you can't change normal text line boxes,
roc: But some boxes controlled by script.
glazou: Some polyfills don't create new boxes, but change existing
ones.
roc: Which properties?
glazou: One example controlling the borders property - multiple
borders around an element
glazou: That impacts the width and everything of the element.
roc: I don't think you should do that by taking a box and changing
it's width - implications on rest of layout is very complex.
roc: There are different ways of doing that (e.g. anonymous DOM
nodes).
glazou: But we don't control what people are doing with polyfills.
roc: As long as we provide some way, we don't need to provide
every way.
plinss: We want to provide a way for polyfills to do all these
things eventually,
plinss: But I agree with roc is that we start with a read-only box
tree API.
plinss: We can add hooks later on.
plinss: The layout system can invoke hooks - 'how big do you want
to be before I create the box tree'.
roc: Yes, exactly.
plinss: Separate processes - start with static.
glazou: OK, yes I'm fine with that. Wanted to make sure we're not
saying we'll never act on the box tree.
roc: Right.
glazou: We closed doors like this 15 years ago - end up discussing
them far too late.
plinss: We may say API surface will always be immutable, but we
may provide other means.
SteveZ: This first step is hard enough without trying to add
mutation now. But we need to think about how we might
mutate in deciding what to expose.
<franremy> FWIW, I don't believe in mutating boxes directly either.
Rossen: I'm having a hard time understanding "what is a box tree?".
Rossen: It sounds like we're talking about the same thing, but I
guarantee we're not.
Rossen: The text in CSS is intentionally vague.
Rossen: It almost conveys nothing.
<bkardell> +1 to begin defining the box tree
<bkardell> At least at a high level you gotta start spelling it
out.
roc: I think that's mainly editorial.
roc: If there's a specific question, people in the room can
probably say the answer.
dbaron: TabAtkins, fantasai and I would independently give the
same answer.
<dbaron> ... most of the time
<franremy> I think there's some confusion here between 'boxes'
(layout by-product) and 'renderable boxes' (which I
call slots) - polyfills may need both for different
purposes
roc: We have a sequence of fragments that are actually nothing
like a box, but I don't think it's hard.
roc: If you give me DOM, I think I can tell you what the box tree
should be.
Rossen: But I think implementations will be different.
roc: Right, I know our implementation doesn't do exactly the same
build of the box tree, but I think we can agree on what the
tree is.
Rossen: Where anonymous boxes come into play I think we have
different internal representation.
roc: But they're in the spec.
Rossen: So is table fixup, but go check interoperability.
roc: But other browsers are buggy and ours is right ;-)
<bkardell> If it is easily defined/mostly editorial how hard
should it be to sketch it out with a little bit of
handwaving over difficult bits here in Sydney so that
you can talk about what to expose
ChrisL: This is why it needs to be a snapshot API - can't go both
directions.
roc: OK, what do we actually need on these boxes?
roc: Geometry, width, height.
roc: What else do people want?
shans: First and last baseline.
dbaron: What created the box - what element it was for, maybe some
specific type of anonymous box.
Rossen: We need to be able to define the types of boxes
Rossen: If I have the table box, it should be reasonable to ask
for columns and rows.
Rossen: Line boxes will have different properties.
ChrisL: Is it true that any given fragment has exactly one box
parent?
roc: A box generates multiple fragments.
roc: For any given fragment it'll have a parent fragment but it is
generated by a box.
ChrisL: Makes sense.
astearns: Do we add properties to these boxes in addition to
width/height - things that went into determining such as
content-min, -max?
roc: Fragments have content, padding, border, outline rectangles.
roc: We might want to expose intrinsic width.
roc: The intrinsics as well as actuals.
johanneswilm: Content of a line box would be a range.
<ChrisL> and they are not (necessarily) axis aligned.
roc: That's a really good question.
roc: We need ranges whose end points can be in anonymous content.
roc: Not good enough to just be a DOM range, need something
slightly better.
roc: But it's a really good point that fragments are going to
cover a range.
astearns: You may have multiple ranges if something within the
range has been floated out of that line box.
roc: You could look at it that way, or I think you could ignore
that problem.
roc: Line has a start and end in content in the DOM,
roc: And you assume everything in between is part of the line,
astearns: Depends on how much script you have to write to figure
out all of those edge cases.
astearns: It might be simpler to have a list of ranges.
roc: Yes.
roc: One more thing - with fragments you can have different styles
for different strings.
roc: If we go this way, we'll want to expose the style for each
fragment.
Rossen: Any group who wants to start defining boxes?
Rossen: I am.
shans: Me too.
shans: I don't have a lot of expertise, but have lots of interest
plinss: Me too.
<iank> Me too.
Rossen: I think we've touched on most of the topics in the agenda
Rossen: what about custom paint?
iank: I talked about this a bit.
Rossen: Time to collect topics and figure out scope,
Rossen: And who is interested in what.
[10-15 min break - back at 3:05 Sydney time]
<franremy> [using the break to react on the interesting "DOM
ranges won't work for anonymous content" comment from
roc: I think coming up with something other than DOM
ranges would delay things a lot; anonymous content is
usually simple so we could reuse standard DOM Ranges
for anonymous content by spanning the range over a
'detached' DOM element like
<div::before>Chapter 1</div::before>; i.e. we create a
detached DOM node
<franremy> that matches anonymous content but isn't attached to
the document (and wouldn't be live-updating); this
should be sufficient to make the polyfills happy]
Received on Tuesday, 3 March 2015 19:20:40 UTC