W3C home > Mailing lists > Public > public-houdini@w3.org > March 2015

Minutes Sydney F2F 2015-02-07 Part III: Houdini-Related Presentations Continued

From: Dael Jackson <daelcss@gmail.com>
Date: Tue, 3 Mar 2015 14:20:11 -0500
Message-ID: <CADhPm3trHTJkuf6BbdjZZB2nB3V_vqeVF6V-MJ6a8-_E3Yyk9w@mail.gmail.com>
To: public-houdini@w3.org
Houdini-Related Presentations Continued
---------------------------------------

  - After lunch, rbyers brought up allowing more controls for
        scrolling, especially to allow extensible scrolling so that
        things like chain scrolling and infinite scrolling can be
        done and explained.
  - bkardell brought up being able to explain and define isolation
        which lead to Rossen leading a conversation about better
        definitions of what breaking means and how to create more
        break controls which has been of especial interest for
        E-Publishers.
  - astearns brought forward the creation of something that he's
        calling "style mutation observers" to trigger an event when
        a style changes.
  - Lastly, Rossen brought up the box tree as something crucial to
        define and decide how to expose in order to make almost any
        of the previous topics possible.  The group leaned toward
        first creating a definition of the box tree in a static
        state and then giving hooks into the box tree to make it
        dynamic.

===== FULL MINUTES BELOW ======

Houdini-Related Presentations con't
===================================

Scrolling Extensibility - rbyers
--------------------------------
  Scribe: shans

  rbyers: I think scrolling extensibility is related to this area
          too.
  rbyers: I'll just paste in a couple of links.
  <rbyers> https://docs.google.com/a/chromium.org/presentation/d/1P5LYe-jqC0mSFJoBDz8gfJZMDwj6aGeFYLx_AD6LHVU/edit#slide=id.g4119c536d_015
  <rbyers> https://www.youtube.com/watch?v=L8aTuoQWI8A
  <rbyers> https://docs.google.com/a/chromium.org/document/d/1VnvAqeWFG9JFZfgG5evBqrLGDZYRE5w6G5jEDORekPY/edit#heading=h.kd0gtwwz5bf9
  rbyers: This is a talk I gave at BlinkOn. Summary: native apps
          have lots of interesting scroll effects, we want to be
          able to do this natively too.
  rbyers: The most extreme example is pull-to-refresh.
  rbyers: Working on a few primitives that might help.
  rbyers: First thing is need some ability to synchronize with
          scrolling.
  rbyers: e.g. if transforming something around the page,
  rbyers: scroll-blocks-on is what we've exposed in canary.
  rbyers: Every browser already has some notion of what scroll
          blocks on, it's good to expose that.
  <rbyers> https://docs.google.com/a/chromium.org/document/d/1aOQRw76C0enLBd0mCG_-IM6bso7DxXwvqTiRWgNdTn8/edit
  rbyers: This property gives you a way to control that.
  rbyers: For example, stop waiting for touch events:
  rbyers: Lots of potential concerns: We want to make sure we don't
          provide a footgun.
  rbyers: It's still very experimental.
  rbyers: I don't want to do anything in isolation, hope we can talk
          about in this group.

  rbyers: Synchronizing with scrolling can be done two ways - either
          by forcing everything to happen on the main thread, or via
          extensible scrolling.
  rbyers: Also, how do we explain things like scroll chaining?
  rbyers: e.g. in pull-to-refresh, the spring needs to collapse
          before anything else scrolls.
  rbyers: We should be able to explain overflow scroll property and
          build it in JS.
  rbyers: I've got a proposal called scroll customization that
          provides custom scroll and custom scroll-distribute
          methods to change how scrolling is implemented.
  rbyers: It's super prototype, but I'm hoping others will be
          interested in talking about this topic.
  rbyers: I'm happy to give demos etc. if people want to talk more.

  glazou: The topic that triggered the original discussions was the
          box tree. We haven't discussed that yet.
  astearns: During lunch we talked about the best venue for this
            discussion.
  astearns: Maybe the Houdini list is better than the style list.
  rbyers: I think that was a slightly different problem - threading -
          but it's related. We probably don't want to talk about
          threading here.

  plinss: With scrolling, in the early days of the gecko engine we
          had a choice of how to describe scrolling in terms of
          internal models.
  plinss: Internally had an anonymous box on the outside and another
          on the inside.
  plinss: But are you moving the layout of these boxes around, or is
          it just something that happens at render time?
  plinss: This is going to be an interesting one - there's lots of
          cases that potentially fall on both sides.
  plinss: This is going to be hard to standardize because of
          differences in implementation but I think we should.
  plinss: If something is overflow: visible and scrollable through
          script, and other things wrap around it live, then that
          needs to happen at layout.

  rbyers: With the scroll customization primitives, if you first
          want to build something like overflow: scroll, then with
          the box model primitives that we're talking about should
          be able to meet some of these without adding extra to the
          platform.
  ChrisL: From a user perspective, some things that look like
          scrolling aren't. e.g. tumblr model - things are actually
          being evicted from the top. But it looks like scrolling to
          the user.
  ChrisL: So decoupling gestures and presentation from back-end
          implementation.
  rbyers: We want to decouple the intent from what that intent
          triggers. That's our thinking too.
  rbyers: Infinite scrolling is a good example, but we can build it
          on top of scrolling.
  rbyers: But and example is what if I want to scroll through a
          spinning wheel and come back to where I started. You can't
          do that today.
  rbyers: We should be able to implement a scroller that under the
          hood is a 3D rotate.
  ChrisL: We also can't do scrolling as navigation mechanism.
  plinss: Infinite scroll is a really good example of why we need
          the box tree. Sometimes infinite scrolling is just
          capturing the gesture. Sometimes it's lazy loading.
  plinss: We should be giving devs primitives to do these properly.
          Should, for example, be able to say that there are 10k
          items, 500px high, without them being explicitly in the
          DOM.
  plinss: You need to get to primitives that let these things happen
          without breaking everything else.

Breaking/Isolation - bkardell and Rossen
----------------------------------------

  murakami: Are those things related to page views included in the
            agenda?
  Rossen: I don't think there's a fixed agenda, but we should
          definitely talk about that.
  <bkardell> No one has discussed isolation.

  bkardell: One thing that I think is worth talking about is
            isolation.
  bkardell: That's really dominated web-apps and www-style.
  bkardell: What we have currently is almost impossible to manage in
            certain scenarios.
  bkardell: Developers really struggle with it. Worth discussing
            what the missing primitive is and where it belongs.
  <bkardell> https://briankardell.wordpress.com/2015/01/14/friendly-fire-the-fog-of-dom/
  <astearns> pagination requirements from digipub group:
             https://www.w3.org/dpub/IG/wiki/Pagination_Requirements

  Rossen: One more topic: haven't heard yet about how we deal with
          breaking, and what breaking means.
  Rossen: Line breaks, box breaks, page breaks (without defining
          pages).
  Rossen: One of the ideas that I wanted to include as part of the
          next level of fragmentation is the ability to have named
          breaks.
  Rossen: We can let people define their own types of breaking.
  Rossen: e.g. I'm defining my own page element and want to know
          when something inside has reached a breaking boundary,
  Rossen: And then handle what happens.
  Rossen: We currently have the names 'column', 'region', 'page' and
          the platform deals with them,
  Rossen: There's no reason why the host shouldn't be able to define
          what the handling is.
  Rossen: Our implementation is pretty much the same and we use it
          for everything.
  Rossen: It's the primitive we use for breaking layout between
          fragments.

  astearns: Can you describe a custom break type use case?
  Rossen: Yes.
  Rossen: We've been getting lots of feedback from applications that
          do EPUB content presentation.
  Rossen: It's really hard to start layout somewhere in the middle
          unless you control the content entirely.
  Rossen: e.g. if I try to load the bible on an 8" device, chances
          are that there will be thousands of pages.
  Rossen: If I want to resume reading in the middle, it's difficult
          to know how to do that without doing layout.
  Rossen: Having breaks and being able to do layout once for that
          size and font set lets us resume without necessarily
          having to chunk the content.
  <ChrisL> break-book break-chapter break-verse
  Rossen: I want named breaks because I'm sick of arguing about
          pages.
  Rossen: It should be whatever you want to call a page.
  Rossen: If I can create a pagination context that understands
          breaking 'foo', then I should be able to generate breaks
          in 'foo' from content and handle them in the 'foo' engine.

  bkardell: Would this have an impact on the URL?
  bkardell: How would you load a page from the middle outward?
  bkardell: Is the bible a single HTML page in your example?
  Rossen: Yes.
  bkardell: There's lots of ways the bible can be divided.
  bkardell: At some level this could potentially effect URLs -
            should it?
  johanneswilm: As I understand it, the entire bible is one HTML
                page. You load it, but first the developer has
                already done a pagination run. So page 316 could
                immediately jump to the right place in the content.
  johanneswilm: So how would this work? Right now I'd count letters
                per page and store it somewhere on the page.
  johanneswilm: Then I'd sum the first 315 pages, see how many
                letters, then jump to the exact point in the text.
  johanneswilm: But how would you store these counts?
  johanneswilm: Should you have to display once? Or should this
                happen without ever having loaded?
  <bkardell> But I can share a link to Genesis chapter 1 vs 3 on the
             web - and that's important.

  Rossen: Today we don't have a primitive that exposes the break
          record.
  Rossen: Usually it's a collection of all the elements and boxes
          that have been broken, plus enough context to describe how
          much content was consumed and how much is remaining.
  Rossen: Without it being exposed you can't just resume generating
          page.

  bkardell: So I can share a link to Genesis chapter 3. But if the
            URL doesn't change, how could you do that?
  bkardell: It feels like this should be holistic.
  Rossen: You should still be able to find the position of a named
          element and resume layout from the nearest page break
          before that element.
  <johanneswilm> +1 for allowing user defined breaktypes. Check this
                 example of how it can currently be done:

http://fiduswriter.github.io/simplePagination.js/simplePagination.html
  bkardell: But I can't share that link if it doesn't have an
            affordance.
  glazou: This isn't about sharing. Concrete example - you have a
          document in your browser, you've scrolled, you close the
          tab, then you reopen it. You're in the same position.
          Everything had to be re-laid-out because that's the only
          way it can work right now. Rossen wants to optimize to
          layout from nearest break. He doesn't want to change
          anything.
  bkardell: If that were a possibility I imagine we'd use it a lot
            more. I'm wondering if that can also be backed out so
            that there are URLs to each page.
  ChrisL: You do this with identifiers in the HTML, then declare
          there are page breaks.
  <bkardell> Thank you, that's what I was asking for.
  ChrisL: You don't share the page (different devices allocate
          differently to pages).
  ChrisL: Instead, it's so that if you go back to the device then
          you get the pages faster.
  ChrisL: But someone else might have different pagination.

  glazou: Why do you need user defined breakage?
  Rossen: You can only break on column, region, or page right now.
          From an application's point of view, if you're creating
          your own paginator and view, then I want to declare which
          breaks I honor. (e.g. I handle article boundaries but
          don't paginate the articles).
  Rossen: So with user defined page breaks I can define what
          'article begin' looks like and that I can break on article.
  glazou: It seems like regions are a solution here?
  Rossen: That's what we thought,
  Rossen: But some cases are a little bit hard.

  <dbaron> What does this have to do with the agenda?
  <bkardell> dbaron: It just has to do with whether the idea loses
             an important quality of the web if you open that door.
  <bkardell> dbaron: I think it sounds like it doesn't necessarily,
             so it's not important.

Style Mutation Observer - astearns
----------------------------------
  Scribe: rbyers

  astearns: I want to discuss "style mutation observers"
  astearns: We need to know when a style property changes.
  ChrisL: Any selector would have an observer which triggers an
          event when what the selector matches changes?
  astearns: Yes.
  <bkardell> jquery.live is essentially this
  <bkardell> http://api.jquery.com/delegate/
  <bkardell> difference with jquery is it is for events

  glazou: Selector matching observer or a property computed value
          changing observer?
  astearns: That's a good question.
  astearns: New property - want to know what selectors it's being
            defined in.
  astearns: When property value changes.
  glazou: OK, so you need both.
  ChrisL: Like DOM mutation, if nothing is watching there's no
          downside, right?
  ChrisL: Not doing anything doesn't cause any downside.
  glazou: True, except you still have to check the number of
          observers.

  dino: Why is your API just about selectors rather than detecting,
        for example computed style changes?
  dino: I assumed "style mutation" could detect when style on an
        element had changed.
  astearns: I'm not sure what all needs to be tracked. What
            selectors match seems to be the most important thing.
  astearns: It will happen much more often than the computed style
            changing,
  astearns: But maybe it's better to track the computed style
            changes per element.
  <bkardell> Maybe you'd set it up like mutation observers - watch a
             selector and optionally only when the style changes

  glazou: You probably really want to know about the rule, not the
          selector,
  glazou: And you want to climb up the cascade.
  glazou: We suggested we add it to the OM a long, long time ago. It
          existed in Gecko. I suggest we add it.
  roc: If you're polyfilling a CSS property, what dino said is right.
       When does a computed value change (including element
       added/removed from document),
  roc: You want that notification for all elements.
  roc: You don't need to know anything about selectors.
  shans: I think webkit had a similar API for walking up the cascade.
         Both WebKit and Mozilla had to remove it for security
         concerns.
  shans: I'm not saying we shouldn't do it, but need to build
         awareness.
  shans: It was an API that gave you a set of style rules that
         matched a given element.
  <bkardell> I think you can build this on mutation observers today
             with just .filter and .matches right? where is the
             security issue?
  roc: You don't need that.
  roc: That's what people polyfilling CSS properties need.
  roc: Editors need other things.

  astearns: Could this just be built on top of DOM mutation
            observers? Triggers on computed style?
  roc: No, that's not how you'd do it.
  shans: I used this once to polyfill CSS events.
  roc: That's different than CSS properties.
  shans: Focusing on narrow use cases like user custom properties
         then you don't need it, but other cases do.

  glazou: I have a concrete example.
  glazou: Editor, style property span on it.
  glazou: Works whatever the style applied to the document - may be
          complex stylesheet.
  glazou: Will try to find the deepest rule to modify.
  glazou: All the machinery is hidden to the user. User just says "I
          want to set the font white here".
  <bkardell> glazou devtools must essentially have this, right?
  dino: Why does that need to be web-facing?
  dino: The devtools API has this.
  dino: Could be standardized outside this group.
  glazou: IMHO it's extensibility of the style engine - so fits here.
  <bkardell> glazou are you essentially saying that you should be
             able to have an element and say "what rules am I
             matching right now?"

  dino: I need to understand what use cases we want to address. roc
        and I are talking about very different use cases.

Box Tree - Rossen
-----------------

  Rossen: Any other topics?
  Rossen: One topic not put forward explicitly is the box tree.
  Rossen: This is the one topic that lured me into all this.
  Rossen: Most of the topics discussed today would require deciding
          what the box tree is and what it's API surface should be,
          then tacking on additional behavior.
  Rossen: But the parser and style side is just as important.
  <bkardell> Parser doesn't require box tree as far as I can tell,
             nor do custom pseudos?

  ChrisL: It's like CSS 2.0.
  ChrisL: Where we said there should be a clear separation between
          the document tree and style tree,
  ChrisL: But one vendor had a single tree and blocked this.
  Rossen: Yes, it's time to change.
  Rossen: The box tree sounds like a fairly major piece that's worth
          discussing.
  Rossen: I would be a good starting point.
  Rossen: I want to hear ideas for how we should expose the box
          tree, what APIs should look like, what hooks should we be
          giving.
  <bkardell> Isn't it necessary to concretely define the box tree in
             an agreed upon way before you start talking about
             exposing it?

  roc: I think it's relatively easy to expose a read-only non-live
       information about the box tree.
  roc: We need to define what it is, but mainly an editorial issue.
  roc: The exact details of getting browsers to match those specs
       may be non-trivial,
  roc: But exposing in a read-only way shouldn't be too hard for any
       of us. It's needed for drawing borders correctly etc.
  roc: It has to be a non-live API-
  roc: A snapshot of the current state in time.
  roc: Different engines don't have these boxes sitting around.
       Tracking changes will be really painful.
  ChrisL: You still want people to change the box model, just not
          directly right?
  roc: Providing mutation should be a separate problem.
  roc: It's best to separate the boxes that are under the browser's
       layout control from those under developer's layout control.
  roc: Simplify things if you can't change normal text line boxes,
  roc: But some boxes controlled by script.

  glazou: Some polyfills don't create new boxes, but change existing
          ones.
  roc: Which properties?
  glazou: One example controlling the borders property - multiple
          borders around an element
  glazou: That impacts the width and everything of the element.
  roc: I don't think you should do that by taking a box and changing
       it's width - implications on rest of layout is very complex.
  roc: There are different ways of doing that (e.g. anonymous DOM
       nodes).
  glazou: But we don't control what people are doing with polyfills.
  roc: As long as we provide some way, we don't need to provide
       every way.
  plinss: We want to provide a way for polyfills to do all these
          things eventually,
  plinss: But I agree with roc is that we start with a read-only box
          tree API.
  plinss: We can add hooks later on.
  plinss: The layout system can invoke hooks - 'how big do you want
          to be before I create the box tree'.
  roc: Yes, exactly.
  plinss: Separate processes - start with static.
  glazou: OK, yes I'm fine with that. Wanted to make sure we're not
          saying we'll never act on the box tree.
  roc: Right.
  glazou: We closed doors like this 15 years ago - end up discussing
          them far too late.
  plinss: We may say API surface will always be immutable, but we
          may provide other means.
  SteveZ: This first step is hard enough without trying to add
          mutation now. But we need to think about how we might
          mutate in deciding what to expose.
  <franremy> FWIW, I don't believe in mutating boxes directly either.

  Rossen: I'm having a hard time understanding "what is a box tree?".
  Rossen: It sounds like we're talking about the same thing, but I
          guarantee we're not.
  Rossen: The text in CSS is intentionally vague.
  Rossen: It almost conveys nothing.
  <bkardell> +1 to begin defining the box tree
  <bkardell> At least at a high level you gotta start spelling it
             out.
  roc: I think that's mainly editorial.
  roc: If there's a specific question, people in the room can
       probably say the answer.
  dbaron: TabAtkins, fantasai and I would independently give the
          same answer.
  <dbaron> ... most of the time
  <franremy> I think there's some confusion here between 'boxes'
             (layout by-product) and 'renderable boxes' (which I
             call slots) - polyfills may need both for different
             purposes
  roc: We have a sequence of fragments that are actually nothing
       like a box, but I don't think it's hard.
  roc: If you give me DOM, I think I can tell you what the box tree
       should be.
  Rossen: But I think implementations will be different.
  roc: Right, I know our implementation doesn't do exactly the same
       build of the box tree, but I think we can agree on what the
       tree is.

  Rossen: Where anonymous boxes come into play I think we have
          different internal representation.
  roc: But they're in the spec.
  Rossen: So is table fixup, but go check interoperability.
  roc: But other browsers are buggy and ours is right ;-)
  <bkardell> If it is easily defined/mostly editorial how hard
             should it be to sketch it out with a little bit of
             handwaving over difficult bits here in Sydney so that
             you can talk about what to expose
  ChrisL: This is why it needs to be a snapshot API - can't go both
          directions.

  roc: OK, what do we actually need on these boxes?
  roc: Geometry, width, height.
  roc: What else do people want?
  shans: First and last baseline.
  dbaron: What created the box - what element it was for, maybe some
          specific type of anonymous box.
  Rossen: We need to be able to define the types of boxes
  Rossen: If I have the table box, it should be reasonable to ask
          for columns and rows.
  Rossen: Line boxes will have different properties.

  ChrisL: Is it true that any given fragment has exactly one box
          parent?
  roc: A box generates multiple fragments.
  roc: For any given fragment it'll have a parent fragment but it is
       generated by a box.
  ChrisL: Makes sense.

  astearns: Do we add properties to these boxes in addition to
            width/height - things that went into determining such as
            content-min, -max?
  roc: Fragments have content, padding, border, outline rectangles.
  roc: We might want to expose intrinsic width.
  roc: The intrinsics as well as actuals.
  johanneswilm: Content of a line box would be a range.
  <ChrisL> and they are not (necessarily) axis aligned.
  roc: That's a really good question.
  roc: We need ranges whose end points can be in anonymous content.
  roc: Not good enough to just be a DOM range, need something
       slightly better.
  roc: But it's a really good point that fragments are going to
       cover a range.
  astearns: You may have multiple ranges if something within the
            range has been floated out of that line box.
  roc: You could look at it that way, or I think you could ignore
       that problem.
  roc: Line has a start and end in content in the DOM,
  roc: And you assume everything in between is part of the line,
  astearns: Depends on how much script you have to write to figure
            out all of those edge cases.
  astearns: It might be simpler to have a list of ranges.
  roc: Yes.
  roc: One more thing - with fragments you can have different styles
       for different strings.
  roc: If we go this way, we'll want to expose the style for each
       fragment.

  Rossen: Any group who wants to start defining boxes?
  Rossen: I am.
  shans: Me too.
  shans: I don't have a lot of expertise, but have lots of interest
  plinss: Me too.
  <iank> Me too.

  Rossen: I think we've touched on most of the topics in the agenda
  Rossen: what about custom paint?
  iank: I talked about this a bit.
  Rossen: Time to collect topics and figure out scope,
  Rossen: And who is interested in what.

  [10-15 min break - back at 3:05 Sydney time]

  <franremy> [using the break to react on the interesting "DOM
             ranges won't work for anonymous content" comment from
             roc: I think coming up with something other than DOM
             ranges would delay things a lot; anonymous content is
             usually simple so we could reuse standard DOM Ranges
             for anonymous content by spanning the range over a
             'detached' DOM element like
             <div::before>Chapter 1</div::before>; i.e. we create a
             detached DOM node
  <franremy> that matches anonymous content but isn't attached to
             the document (and wouldn't be live-updating); this
             should be sufficient to make the polyfills happy]
Received on Tuesday, 3 March 2015 19:20:40 UTC

This archive was generated by hypermail 2.4.0 : Friday, 17 January 2020 19:53:22 UTC