[Houdini] Minutes Sydney F2F 2016-01-31 Part II: Layout API

=======================================================
  These are the official Houdini Task Force minutes.

         Unless you're correcting the minutes,
        please respond by starting a new thread
           with an appropriate subject line.
=======================================================

Layout API
----------

  - iank introduced the group to the work he's done on developing
      the spec (text at the meeting was here:
      http://bfgeek.com/css-houdini-drafts/css-layout-api/Overview.html
      It has since been made public:
      https://drafts.css-houdini.org/css-layout-api/)
      - registerLayout gets a class, is instantiated one per box,
           and have two callbacks; one for layout and one for
           computedStyle
      - inputProperties gets a breakToken and a constraintSpace.
          There's also childInputProperties that function similarly,
          but just on the children.
      - The layout function is similar but for ContentObject and
          returns the traits { width, height, List<Area> children }.
          Children is the list of areas you've produced as your
          content objects.
      - When passed a breakToken in layout you start producing
          fragments from that breakToken and the breakToken should
          give you all the information you need to do so.
      - constraintSpace takes the values of width (inline), height
          (block), and exclusions. Width and height can be either
          definite or null (infinite). Exclusions are a list of
          places where you shouldn't layout.
  - Concern was expressed about how to handle min/max content and
       will be marked as a large issue in the spec.
  - Handling character direction was a gap in the spec that
       concerned some, but there were several opinions that bidi
       resolution should be out of scope for the first version of
       the spec, especially since the information in doLayout isn't
       the only information you receive in layout.
  - doLayout will return a promise in order to allow for parallel
       layout, though it will require more investigation to ensure
       it is performant.
  - constraintSpace received some concerns about scrolling and
       fragmentation for this property. Describing scrolling was
       decided to be out of scope for this spec. Fragmentation
       should be handled in the text and could be with some adjustments.
  - An additional concern about constraintSpace is that it was
       written without display: inside/outside. Though the intention
       was to roll back unintentional behavior, several people felt
       that the behavior was needed.

===== FULL MINUTES BELOW =======

Agenda: https://github.com/w3c/css-houdini-drafts/wiki/Sydney-F2F-January-2016

Scribe: ojan

Layout API
----------

  <iank> http://bfgeek.com/css-houdini-drafts/css-layout-api/Overview.html
  iank: I wrote up rough spec,
  iank: just on personal website,
  iank: but don't read it...it's confusing.
  iank: I will talk through it now.

  iank: The concept of layout is that we have a registerLayout
        function.
  iank: It's similar to paint,
  iank: give it a class,
  iank: gets instantiated once per area/box/thing,
  iank: you get two callbacks.
  iank: First one is layout.
  iank: In layout what you get given is a list of your children,
  iank: you get given your own computedStyle.

  iank: inputProperties you said you'd invalidate on appear in the
        computedStyle stylemap
  iank: e.g. flex.
  iank: You get given a breakToken, which I'll explain in a minute,
  iank: and a constraintSpace.
  iank: layout(List<Area> children, styleMap, breakToken,
        constraintSpace).
  iank: In addition to inputProps, you list childInputProps,
  iank: e.g. in flexbox I require flex-shrink on the children.
  iank: inputProperties show up in the styleMap,
  iank: the childProperties show up in the child Area's styleMap.

  iank: An Area can perform doLayout, which takes a constraintSpace
        and an optional breakToken.
  iank: Area {
  iank: styleMap;
  iank: doLayout(constraintSpace, breakToken);
  iank: }
  <dbaron> My biggest concern so far is intrinsic size computation
           and layout being in the same callback
  <dbaron> well, maybe the biggest concern is that this draft should
           get in the official repository rather than being in Ian's
           own repo...

  iank: For layout function s/Area/ContentObject/
  iank: for the list of children.
  iank: Layout returns traits that consist of { width, height,
        List<Area> children }.
  iank: Children is the list of areas you've produced as your
        content objects.
  Florian: Are you going to do line breaking?
  iank: Yes, will get to that.
  iank: This is all in the spec, but just explaining it here.
  iank: layout(child, constraintSpace, breakToken, styleMap)
  iank: In layout you iterate over your children, create areas from
        them and then you return the generated areas,
  iank: this is basically fragmentation.
  iank: for (child in children) {
  iank: var area = child.doLayout(...);
  iank: }
  iank: Return list of areas.

  Florian: I think I see where this is going in regard to
           fragmenting children, but how about fragmenting yourself
           and re-layouting children then?
  iank: This gets into breakTokens
  iank: In layout you get given a breakToken that you can pass to
        your children that says "continue doing layout starting at
        the breakToken"
  iank: <drawing example>
  iank: First doLayout call gives you your first line with an area
        and a breakToken saying when it last broke.
  iank: Then you feed that breakToken back into doLayout so that you
        can do the next line.
  Florian: Does this mean fragmentation has to be greedy and you
           can't do balancing?
  Rossen: You can do whatever algorithm you want with backtracking.
  Florian: If you want to do balancing you have to do multiple
           passes?
  iank, Rossen: yeah
  shane: Yes, fundamentally this is why balancing is expensive
         because you have to do multiple passes. not fundamental to
         the API.
  iank: The only constraint is that the children areas you return
        must be generated from a contiguous set of breakTokens.
  SteveZ: So a break token is a point in the source string?
  iank: It doesn't have to be in the source string.
  iank: Here's my naive sense of what a breakToken should contain;
  iank: There's one that's completely opaque. it's just for feeding
        back into doLayout.
  iank: The other breakToken (passed into layout) has two things
        ContentObject and has the opaque breakToken from the
        doLayout methods.
  Rossen: There needs to be some kind of trait there as well.
  Rossen: For example if your content object was an element then you
          need to know what kind of trait the element produced.
  Rossen: If you have a type properties that was specified and you
          took x amount of that property.
  Rossen: That should be expressed as a trait.
  iank: It probably needs to be flexible to other data in the
        breakToken.
  dbaron: You're doing in place switch to the area terminology.
  dbaron: But what you have in the draft uses box and fragment, and
          does so correctly.
  dbaron: And it actually needs to be that way and it's doing it
          right,
  dbaron: can we stick to box and fragments.
  Florian: This isn't entirely on the right side.
  dbaron: Boxes and fragments are
  iank: Lets not bikeshed this now.
  <general outrage>

  iank: breakToken contains a box.
  iank: When you get given a breakToken in layout you have to start
        producing fragments from that breakToken and the breakToken
        should give you all the information you need to do so.
  <dbaron> (I don't follow the break token stuff yet.)
  Florian: If you're thinking about doing flexbox for this, is it in
           the flexbox or flex items that you consider things like
           the min-content size?
  Florian: min-content size needs to be taken into account when
           you're laying out the entire thing.
  iank: Pretend a fragment has a min-content/max-content associated
        with it
  iank: In the spec see all the things fragment has on it. width,
        height, min/max content, ignore un-positioned fragments for
        now, opaque breakToken and ignore baseline for the moment.
  <dbaron> Also, I think min-content and max-content are per-box
           rather than per-fragment... at least for many types of
           fragmentation

  astearns: Should fragments also have a notion of the content they
            contain?
  Rossen: Each box is associated with a content element. If you want
          to know what's inside of you you can go to your collections.
  astearns: In this example where you've broken the line I'd like to
            find the first line and find the range of content in the
            line.
  iank: You want to get the string out.
  astearns: Yes. might be more than a string.
  iank: I haven't added it in because I don't know how it would work.
  Rossen: Why would you need it at this level?
  astearns: I was thinking of pagination case where you have content
            fragmented across pages.
  astearns: And you have footnotes that need to be laid out on that
            page.
  astearns: It's the reason we have range sequence on the regions API.
  Florian: Wouldn't the footnote be one of the boxes in the API?
  dbaron: Depends how we define out of flow things in the box tree.
  iank: Let's avoid this detail for now.
  astearns: Also, as you're laying out do you need to make
            distinguish between fragments that have laid out and
            fragments that are in overflow.

  SteveZ: Related to astearns's question, how would I do first line?
  SteveZ: I need to know what content after I've done capital first
          letter first on first line.
  Rossen: That's in your output breakToken. It tells you where you
          ended.
  Rossen: I don't need to know how far it got, just need to know
          that when you're inside the layout.

  dbaron: I'm concerned about a few things related to min/max
          content sizes.
  dbaron: One is that I don't like the way it's doing them in the
          same layout callback as the actual layout computations for
          two reasons.
  dbaron: 1. Implementations currently implement it as a pass
          separate and before layout.
  dbaron: 2. In most breaking situations min-content/max-content are
          per-box instead of per fragment.
  <general discussion about how blink/webkit/gecko do as separate
       pass and edge does not>
  Florian: For #2 you can ask the entire thing for it's min-content
           and then fragment as you move along.
  iank: Min and max content are computed when you give a null
        constraint space saying you have infinite size for layout so
        don't fragment
  iank: The min/max content here is from that calculation.
  iank: Lets leave that as a large issue.
  Florian: Adding to it, in rare cases, min/max content depend on
           the position.
  Florian: I'm referring to line snapping.
  Rossen: e.g. when you have floaters and have to take them into
          account?
  Florian: No, I meant line grid.
  dbaron: We haven't even defined how min/max content work in those
          cases and would try to avoid having them affected by
          position.
  iank: I will leave how we compute min/max content as an larger
        open issue.
  iank: Fragments don't position themselves, the parent can position
        them in it's space.

  SteveZ: Can you ask where the baseline is?
  iank: Yes. It's a read only attribute on the fragment.
  iank: That was obvious and added it for the moment.
  Florian: Can you ask where the baselines are?
  iank: Just make it a list?
  SteveZ: It's not so simple.
  Florian: If you get dominant baseline, that's enough for a start.
  <astearns> needs to be much better defined, but it's nice that
             it's there in the first draft :)

  iank: Back to what you return from the layout method...
  iank: Width and height of fragment, list of children fragments you
        contain (and you've positioned them)
  iank: They need to be from a contiguous set of breakTokens.
  iank: The give back a list of unpositioned fragments,
  iank: and also where your baseline is.
  iank: This is how you can propagate baseline up into parents.
  iank: Unpositioned fragments is how you do abspos, fixed, floats
        and stuff like that
  iank: Your layout says I don't know how to position this fragment,
        I just pass it up to my parent.
  iank: You get this eight from the fragment you generated from
        doLayout or the fragment has a list of
        unpositionedFragements you can pass up until it hits
        something that can position them.
  Florian: Depends how we define boxes/fragments and whether they
           are your children.
  iank: It's useful if you want to be able to flatten your layout.
  iank: A complex constraint based layout can artificially flatten
        by passing fragments up without positioning them.

  Bert: I'm interested about breakToken.
  iank: It's up to your parent to either return a breakToken saying
        I still have stuff to layout or call layout on you again to
        get the next fragment.
  Florian: Is it possible for the parent to tell when you need to
           get a different break point because if you need to go
           beyond it you will overflow or fragment?
  iank: Yes, that's in the constraint space.
  Bert: min and max content are in inline direction?
  Florian: Not both?
  iank: No.
  Bert: Not sure it's needed here
  Rossen: Your min content in the block direction is the result of
          the max content of the page.
  Rossen: Assuming you're using your layout engine to compute the
          max,
  Rossen: block direction you need to actually do layout to compute
          min/max.
  Rossen: The other dimension is you run layout with 0 width and
          read the height.
  Florian: Depends on if we go with min/max being separate from
           layout.
  iank: That's only for inline min/max

  iank: Final thing you get is...this is a super dumb API...
  iank: The constraintSpace
  iank: Conceptually has 3 things on it;
  iank: width (inline), height (block)
  iank: can be definite or null.
  iank: null == infinite space in that direction.
  iank: It can have definite in one direction and infinite in the
        other.
  iank: Lastly have a list of exclusions.
  iank: It's super dumb at the moment; there's probably better way
        to describe this.
  iank: Exclusions is areas you shouldn't position fragments in,
  iank: floats for example.
  iank: When you layout and position something it becomes part of
        your exclusion area so that when you layout the next thing
        it's in your exclusions space.
  iank: It doesn't prevent doing something less general.
  iank: child uses constraintSpace just to figure out width/height.
  iank: Parent uses it to say where fragments should be positioned

  Florian: Have you thought about how shapes work with that?
  iank: No.
  iank: Could be an exclusion...not sure if good idea.
  Rossen: Why not?
  iank: Sure. exclusionArea and exclusionRect is a subclass.

  iank: That is it.
  iank: I have no idea how to do overflow.
  dbaron: What do you mean by overflow?
  iank: If you should scroll or not.
  Rossen: We're doing layout right now not scrolling.
  <rossen uses magic fingers>
  Rossen: When we are going through layout and taking off chunks of
          space for various reasons, the one question that came up
          in different contexts...
  Rossen: The first one is that I might have special lines that when
          I'm crossing I might have to take action.
  Rossen: One of them is for fragmentation.
  Rossen: If this is meant for fragmentation there will be a set of
          commands you would have to take into account whatever your
          layout logic is for that.
  Rossen: In the case of overflow, you have a different type of
          constraint.
  Rossen: That says "if I'm crossing this boundary, you don't have
          to stop. It's up to you, but I will be requiring a
          scrollbar."
  Rossen: It's up to the layout logic of the owner of that
          constraint space to keep going or not.
  jet: Such things aren't breakTokens introduced in the middle of
       layout.
  dbaron: Scrolling and fragmentation are different things.
  Rossen: Yes.
  dbaron: Scrolling is something we don't need information in this
          API to handle,
  dbaron: or not in V1.
  dbaron: As long as the thing you're trying to write with the API
          isn't overflow:scroll it seems there is default handling
          already in the implementation.
  dbaron: Maybe we don't want to explain that in V1.
  dbaron: For fragmentation there might need to be more information
          about how you're trying to fragment.
  dbaron: You could be in a space where you want to fragment stuff
          that doesn't fit vertically,
  dbaron: or that doesn't fit horizontally,
  dbaron: and you need to pass that to descendants so they know to
          fragment.

  Rossen: One of the points of confusion about constraintSpace,
  Rossen: there are two types, unbound and bound.
  Rossen: That the space is bound doesn't mean you have to fragment.
  Rossen: When you do layout we are going to the descendants to
          figure out the overall union so you can figure out your
          scroll content area and propagate that out.
  Rossen: And your constraintSpace can say a given line is special
          for fragmenting,
  Rossen: and they will nest and be affected that way.
  Rossen: The unboundness of space is on attribute and fragmentation
          vs non-fragmentation is a different thing.
  Florian: Fragmentation might not just be a boolean, e.g. break
           after column or break after page.
  SteveZ: You said you were doing it in x and y, don't you need to
          pass the layout directions.
  iank: It should be handled by the engine.
  Rossen: You'll always use logical coordinates.
  SteveZ: I want to switch at some point.
  Rossen: Do it on the ConstraintSpace.
  SteveZ: relying on the contentObject to say what direction it's in?
  Rossen: Yes.
  Rossen: Space has a direction. It's always left to right.

  Florian: How do you inform the children that base directionality
           is left-to-right or right-to-left?
  dbaron: One thing here might be that this is not attempting to
          explain inline layout.
  Rossen: Why not?
  iank: I didn't get into text runs here.
  iank: I don't get into positioning characters so bidi handled
        magically.
  Florian: Can you separate the two?
  Rossen: If you're breaking a bunch of divs, do you care what the
          layout inside of the divs are?
  Florian: When you call doLayout on the linebox, this is not one
           you can override?
  ojan: Not in V1 for doing character by character layout, but
        that's a thing you can add to this general structure of API.
  dbaron: Part of what Florian was bringing up is that if you're
          doing inline layout, how that works deeply within the line
          is a function of the bidi parameters getting passed through.
  dbaron: That stuff is reasonably complicated, I'm not sure we want
          to try to attack all of that in the first version.
  dbaron: Last time we talked about how a custom layout would always
          be a block formatting context.
  dbaron: So that we don't do inline layout, float interactions,
          margin collapsing.
  ojan: My perspective: this attempts to have an overarching thing
        that we could later add line layout and float exclusions into.
  iank: But for v1 we could blockify everything.
  ojan: What I was going to say -- there's a strict subset of this
        API that does the block part of layout -- it's not a
        different API, but a subset.
  ojan: This lets us have a single API for block and line layout,
        but we ship the block subset in v1.

  dbaron: I don't understand the breaking model. My gut feeling is
          that it's too simple to possibly be able to work, but
          haven't worked out the details.
  Florian: To have this API work in level one with explaining
           everything don't we need to add more for the bidi stuff?
  iank: These will all be additions to the constraints?
  ojan: dbaron, is the constraintSpace you pass in and a result from
        layout you pass up not conceptually enough?
  dbaron: bidi resolution and reordering are two separate steps.
  dbaron: Resolution goes before fragmentation.
  dbaron: Reordering is after fragmentation and very simple.
  dbaron: I don't see how you explain just that in this model.
  dbaron: If you have the resolution results, doing the reordering
          doesn't seem hard.
  dbaron: The next question is what an extension of that is. Do
          people want to extend it? If so, in what way/direction and
          do we want to make it open or hackable or whatever.
  eae: Some of these things can happen before the calls into these
       layout methods. whether they need to be exposed is up for
       debate. Many use cases you can satisfy without overriding
       bidi behavior.
  Florian: There's a distinction between the things we expose and
           pass through.
  Florian: Authors call the outer loop and need to pass the
           information into the inner loop.
  iank: This is why we have both layout and doLayout.
  iank: That way the engine can pass more information down.
  ojan: Author script doesn't call layout, it calls doLayout.
  Florian: That makes a big difference.
  Florian: So the constraintSpace you pass when you call doLayout
           isn't necessarily the one your receive in layout.
  eae: The browser can pass special information down in the
       constraintSpace.
  ojan: doLayout/layout also lets the browser to caching in terms of
        when to skip re-layout of a subtree because nothing has
        changed.
  Florian: In fact, might want to tell authors explicitly not to do
           caching since the browser has opaque information the
           author doesn't know about.
  iank: This is not all the information available to the engine,
        just a view into the box exposed.
  ojan: I've been thinking that bidi resolution phase is not part of
        this API at all. but I don't know anything about this space,
        so I might not be right.
  dbaron: Makes sense.

  plinss: What about parallel layout?
  iank: I want to make doLayout return a promise.
  ojan: I don't want to do that.
  plinss: Putting entire layout into author space and precluding
          parallelization, you need to leave the loop in engine code.
  iank: I don't want two phases.
  iank/Rossen: We can do it with a promise.
  plinss: Then it need a single promise for all children.
  iank: Some things are inherently sequential.
  Florian: What's wrong with doing the loop at the browser level?
  Florian: So it can run serially or in parallel?
  iank: That's what promises are for.
  iank: You can do "let frag = away doLayout" or await*[doLayout,
        doLayout].
  tantek: There are things that are optimistically parallel because
          they might not depend on each other and after the fact you
          fix up if they did.
  iank: With a promised based API, authors can do that.
  ojan: My only concern with promises is performance overhead.
        Promises in engines today have way more overhead than a
        function callback. I don't know if that's fundamental.
  tantek: Not all things are parallel or sequential, you can
          optimistically layout in parallel.
  tantek: Or some traits can be figured out in parallel (e.g.
          widths, but not heights).
  iank: Yes, to go down that path we'd need to make it promised based.
  iank: We need to experiment with performance of promises.

  Florian: For what tantek is saying, need promise per trait. That's
           way worse overhead.
  iank: We don't want to do that.
  iank: One way to solve that is to doLayout on a bunch of your
        children and then you can call doLayout again with different
        constraints.
  tantek: Yes, that's more what I'm talking about.
  iank: I think we can do that.
  Bert: Doesn't that require that the layout function has the
        possibility to fail?
  Bert: If there were dependencies we didn't know about before you
        have to return failure.
  tantek: Not failure, just return a result that says some aspects
          of this result have a dependency (e.g. vertical
          coordinates).
  Rossen: Nothing precluding doing multiple passes of doLayouts.
  Rossen: That's the error handling.
  Rossen: It's not really errors.
  Rossen: For all practical purposes the layout is done. It's a
          result. You just might not like it.

  Florian: How do you attach an implementation of your layout to a
           thing?
  iank: It's really dumb at the moment. It just uses display.
  <ojan> display: layout('foo')
  <ojan> display: inline-layout('foo')
  <astearns> http://bfgeek.com/css-houdini-drafts/css-layout-api/Overview.html#layout-notation
  Bert: Do the built-ins have names?
  iank: No.
  iank: Similar to paint functions how you cant refer to linear-
        gradient.
  Florian: How do you get the right thing called on flex items?
  iank: They still get coerced to block.
  Rossen: Scenario: Have an element that has a custom layout display
          value: spiral-flexbox. Only I know how the layout works
          and what the rules on the items inside are.
  Rossen: For my purposes I don't care what the items do. So you are
          taking over all the display outside of your children.
  ojan: Let's say you have a flexbox with a display:inline-layout
        ('foo') in it. Does it get changed to layout('foo')?
  ojan: The point is that layout() vs. inline-layout() call the same
        code, just different display-outside. In a flexbox display-
        outside is coerced to block.

  iank: Writing this ignoring display inside/outside.
  Rossen: If I have a table, and it's inside grid or flex, ....
  ojan: Don't do table.
  Rossen: Lets do table!
  ojan: Lets not talk tag names, you put a display: layout into an
        inline.
  dbaron: It triggers the block splitting from CSS 2.1.
  Rossen: I think we should do that.
  Rossen: Overriding your display-outside doesn't make sense, you
          don't know where you get used.
  dbaron: In some contexts it does make sense.
  ojan: There's a world where, if we redesigned the platform from
        scratch, display-outside comes from parent and is not
        settable.
  ojan: Can we do that here instead of having layout and
        inline-layout?
  dbaron: I disagree that the parent should always decide.
  dbaron: That's saying no one ever needs to distinguish between
          inline block and block.
  dbaron: People do want that though.
  ojan: I agree people do that, if we had a platform where you
        couldn't I don't think it's hard to work around.
  Rossen: Kind of, I think that view is very block oriented.
  Rossen: If you have a thing in a flex or grid, what about those?
  dbaron: Yes, those override the display on their children.
  Rossen: Your layout of the parent will layout in whatever context
          it's operating.
  Rossen: For example table row can only handle cells, flex can only
          handle cells.
  ojan: I think a distinct case where developers do something
        different is a block inside an inline.
  ojan: For example, a block or inline block inside an inline,
        developers might want to decide, but if we designed the
        platform from scratch we wouldn't allow that.
  dbaron: The sequence of block or inlines inside a block, that's
          also different.
  dbaron: You could have different display type.
  ojan: Yeah.
  Rossen: We have "flow layout", basing all our decisions on this
          document centric layout ideal seems bad.
  dbaron: When we have display types like this we have an inline and
          a block variant, so you can use them inside blocks.
  dbaron: It doesn't matter outside block, but we still do it.
  Rossen: yes.
  ojan: There's an alternative, we could add a block that contains
        block, and block that contains inlines.
  tantek: We have block that contains inlines, that's block.
  ojan: No, that one is magic.

  dbaron: If we wanted those we need two new types, they're
          explicitly incompatible with blocks today.
  ojan: If we only had layout, what do we lose?
  dbaron: You lose the ability to put it in an inline, you end up
          adding wrappers.
  ojan: ...
  Florian: Yeah.. unpleasant but it's been that way for a while.
  dbaron: I'm hesitant that we should litigate this again, once we
          have inline-flex, grid table etc.
  ojan: This gets to the original thing that Rossen didn't like with
        an inline thing in a block only context it get coerced.
  Rossen: Let's step back.
  Rossen: Why does this matter for your callback?
  ojan: It should call the same code.
  Rossen: It will execute the same code, you just get a
          constraintSpace (layout vs inline layout), depending on
          the parent you might get a different constraintSpace,
          otherwise the callback is the same.
  Florian: Assuming an information heavy constraint space.
  Rossen: Well, you have exclusions, etc.
  Florian: Yeah, baselines, etc.

  Scribe: ojan

  esprehn: I think you need them both and it just controls what
           contraintSpace you get. If you put a custom layout thing
           in a paragraph, might want it inline-block-like...
           shouldn't need to add a wrapper span. In a block, you
           should get a block constraintSpace.
  ojan: What's different bout the constraining space?
  esprehn: In a block, parent gives you your width.
  esprehn: In a inline-block, the contstratingSpace tells you to
           shrink-to-fit.
  dbaron: It also applies shrink-to-fit to blocks as well.
  esprehn: Yes, just a property of your constraintSpace.

  dbaron: For the next steps, can we take this document off your
          personal website?
  dbaron: So that ten years from now when someone reads these
          minutes someone can map the minutes to the document.
  iank: I will do so after lunch.
  iank: Any other next steps?
  Florian: Ship an implementation!
  <everyone is loopy from low blood sugar>

  <br type=LUNCH>

Received on Sunday, 21 February 2016 20:03:56 UTC