[CSSWG] Minutes Sydney F2F 2016-02-02 Part I: A report on research into automatic layout, CSS Text [css-text]

=========================================
  These are the official CSSWG minutes.
  Unless you're correcting the minutes,
 Please respond by starting a new thread
   with an appropriate subject line.
=========================================


A report on research into automatic layout
------------------------------------------

  - Bert had visited groups from Monash University and YesLogic to
      hear about the work they're doing creating algorithms in
      constraint spaces. Their algorithms not only take the known
      constraints, but can recompute as they get additional
      information.
  - Uses for constraint solving include footnotes, sidenotes,
      indexing, table layout, and line breaking. It was mentioned
      some of this could be hooked in via Houdini.
  - There was interest in working more with both groups, especially
      the expertise from the YesLogic folks. Specifically it was
      thought they would have useful feedback on the Grid spec.

CSS Text
--------

  - There was agreement that there is a strong use case to solve for
      aligning to a particular character in table columns.
  - RESOLVED: left-align the alignment characters

===== FULL MINUTES BELOW ======

Agenda: https://wiki.csswg.org/planning/sydney-2016

Present:
  Rossen Atanassov, Microsoft
  Takao Baba, BPS
  L. David Baron, Mozilla
  Brian Birtles, Mozilla Japan
  Bert Bos, W3C
  Bogdan Brinza, Microsoft
  Rick Byers, Google
  Tantek Çelik, Mozilla
  Dave Cramer, Hachette Livre (via telephone)
  Emil A Eklund, Google
  Elika Etemad, Invited Expert
  Simon Fraser, Apple
  Daniel Glazman, Disruptive Innovations (via telephone)
  Jihye Hong, LG Electronics
  Joone Hur, Intel
  Dean Jackson, Apple
  Brian Kardell, jQuery Foundation (via telephone)
  Brad Kemper, Invited Expert
  Ian Kilpatrick, Google
  Peter Linss, Hewlett-Packard
  Cameron McCormack, Mozilla
  Edward O'Connor, Apple
  Simon Pieters, Opera
  Xidorn Quan, Mozilla
  Francois Remy, Microsoft
  Florian Rivoal, Vivliostyle
  Andrey Rybka, Bloomburg
  Hiroshi Sakakibara, BPS
  Simon Sapin, Mozilla (via telephone)
  Hyojin Song, LG Electronics
  Elliott Sprehn, Google
  Alan Stearns, Adobe
  Shane Stephens, Google
  Ojan Vafai, Google
  Jet Villegas, Mozilla
  Philip Walton, Google
  Levi Weintraub, Google
  Greg Whitworth, Microsoft
  Steve Zilles, Adobe Systems

Regrets:
  Tab Atkins, Google
  Dongwoo Joshua Im, Samsung Electronics Co.
  Dael Jackson, Invited Expert
  Chris Lilley, W3C

Scribe: fantasai

A report on research into automatic layout
------------------------------------------

  Bert: I was in Melbourne the last week. Went to see 2 different
        groups of people.
  Bert: First group is people from Monash University, especially Kim
        Marriott.
  Bert: He's professor at Monash, works on constraint solving etc.
  Bert: Met some colleagues as well.
  Bert: And met people from YesLogic, Michael Day and Peter Moulder
        who used to be at Monash.

  Bert: Talked about many things,
  Bert: What they're doing at Monash.
  Bert: They do document engineering,
  Bert: But much more than that,
  Bert: Algorithms for constraint solving and optimizing.
  Bert: Interesting equipment, 3D Printers, laser cutting, 3D
        environments,
  Bert: Can make many things and animate them with computers.
  Bert: Interesting parts for me were things for constraint solving
        and optimization.
  Bert: Trying to explore semi-automatic constraint solving.
  Bert: Number of constraints, computer finds a good solution.
  Bert: These problems are hard, can't do an exhaustive search.
  Bert: In practice many can be solved fairly well this way.

  Bert: One problem was a cancer treatment, placement active charges
        on needles.
  Bert: Before that make 3D model of the area.
  Bert: Visualized in real time, you can see the computer finding a
        set of points where you should stick in needles.
  Bert: It's interactive as well.
  Bert: So if you see that some needles are not where you would put
        them, or you have some extra knowledge not captured in
        constraints, can drag and drop needles around, and it will
        recompute solution based on this new constraint.
  Bert: Variant of traveling salesman problem.
  Bert: Computer finds optimal route.
  Bert: But maybe you didn't think of before, e.g. driver get sick,
        road is blocked, certain driver can't go a certain place, etc.
  Bert: Solve those.
  <gregwhitworth> worth looking at for this:
http://marvl.infotech.monash.edu/webcola/

  Bert: Also thought about applying to layout problems,
  Bert: Brainstormed about what an editor would look like.
  Bert: What are things you would want to influence?
  Bert: Want certain constraints, number of articles to be put on a
        page.
  Bert: But not happy with results, might might want to drag things
        around a bit.
  Bert: Couldn't think of any editor that worked that way.
  Bert: Maybe Daniel wants to build in constraint solvers into his
        editor? :)
  glazou: Get me funding, and no problem!
  Bert: This assumes the editor interactively and in real time
        computes the layout.
  Bert: What would the output be, though?
  Bert: Does the computer output the final set of constraints, and
        browser recomputes?
  Bert: Or should the editor not assume the computer has the same
        capabilities?
  Bert: This interactive manipulation is a way to create constraints
        without using syntax.

  Florian: Don't know how much they looked into it, but if we're
           thinking about CSS and the attempt that we generally make
           for it to be robust in varying environments
  Florian: These manually added constraints, have they thought about
           how to express that when not in authoring environment?
  Bert: No, they've done some demos of .e.g. table algorithms.
  Bert: Table layout could be very fast and always better than the
        current layout.
  Bert: But haven't tried to think about syntax lately, just about
        algorithms.
  Florian: My question isn't about syntax, but these manually added
           constraints? What are they expressed in terms of? When
           the screen size changes, what happens?
  Florian: One reason it's hard to make a good CSS editor is that
           you start with a semantic layout.
  Florian: How do you express these constraints that's not just
           wysiwyg?
  Florian: If you make wysiwyg, it maps poorly to CSS.
  Bert: They haven't done any work on that.
  <gregwhitworth> http://gridstylesheets.org/guides/ccss/
  Bert: But there is something from other people they worked at in
        the past, Washington University in US, ?? and ??? constraint
        cascading style sheets.
  Bert: Using a syntax made by Apple for the internal UI.

  glazou: It is not difficult to write a good CSS editor that is
          strictly visual.
  glazou: Outputs layout based on mouse-defined rendering.
  glazou: What is extremely difficult is taking an existing document
          with existing stylesheets, and add stuff like that.
  glazou: That's why this is complex.
  glazou: That's why you have so many editors outputting HTML+CSS,
          but not using it as their native format
  glazou: Other than that, fairly doable.
  Bert: Michael Day of YesLogic has been thinking of how to use
        things like that
  <astearns> also why you have editors that do not take arbitrary
             HTML+CSS as input
  <glazou> astearns absolutely
  <Florian> glazou: I'll have to have a longer discussion with you
            about this, you know more and I have a ton of questions

  Bert: In a regular language, could say e.g. box a is zero or more
        copies of box C followed by box D
  Bert: Something like that.
  Bert: Usable by people to express how they want the layout to be.

  Bert: We had a thought, also mentioned there, if Houdini works
        well,
  Bert: It might be possible to keep the same underlying system,
        Houdini, but offer alternative ways to express your
        stylesheet
  Bert: Doesn't have to be CSS syntax.
  Bert: Gets translated down to same base level as Houdini.
  Bert: Might be a way to get away from current constraints of CSS.

  Bert: Talked about some of the difficult things in layout:
        footnotes, sidenotes, spreads of pages.
  Bert: Differences between superficially similar things: footnotes
        vs. sidenotes vs. floating images.
  Bert: Similar layout -- float to a side -- but different
        constraints.
  Bert: Floats can only go later, can't go higher.
  Bert: Sidenotes are a bit like normal floats. But don't have to be
        same height or lower, could be higher on the page as well.
  Bert: If you have a spread.
  Bert: Sometimes don't need to put a float on the same page. Could
        put on previous page -- it's still visible anyway
  Bert: How to formalize such constraints?
  Bert: Same thing for line breaking.
  Bert: Maybe not use a single algorithm, but plug in different
        algorithms to do better or faster job.
  Bert: Apply different rules for a particular application.

  Bert: Given Houdini being prepared, maybe it becomes possible.
  Bert: Someone mentioned during lunch that, just occurred the day
        before, found himself reading the same line over and over
        again until he realized both lines were ending with the same
        word.
  Bert: Maybe you want an algorithm that avoids situation like that:
        same word ending two lines.
  Bert: If it doesn't work quite with just changing spacing/line
        breaking, would be nice to have possibility to change the
        text.
  Bert: If the author is available, you can ask author to rewrite
        text to fit better. But on the web, author is not available.
  Bert: But maybe allow alternative text in the document.

  Florian: This also came out in a11y discussion.
  Florian: Some people have difficulty with metaphors, maybe you
           have alternative texts one literal one figurative.
  Florian: Multiple reasons to be interested in this other than
           layout itself.
  <glazou> +1 to what florian said, including linguistic differences

  Bert: Michael Day was interested in development of his market, PDF
        renderers.
  Bert: Looks like he would get more competition.
  Bert: He's happy about that, proves it's interesting and market is
        getting bigger.
  Bert: Makes idea of outputting to paged media / PDF more popular.
  Bert: But was wondering what competitors are working on?
  Bert: Antenna House, Vivliostyle.
  Bert: We noticed another one, existed for awhile but wasn't as
        good in the past, German company,
  Bert: Improved a lot.
  Bert: Competitor to Prince now.

  Bert: Looked also at indexes.
  Bert: If your output isn't interactive, you need things like ToC
        and alphabetical index.
  Bert: They want to do that, and at the moment they are exposing a
        JS interface so you can write a program that creates an index.
  Bert: But the index is created before the layout is done.
  Bert: So you can create the index, sort the index, placeholders
        for page numbers which get resolved after layout.
  * glazou notes that we can't say an element is a link through
           CSS...and we badly need it IMHO
  Bert: But then want to do fixup, like replacing duplicate numbers.
  Bert: Or maybe need JS that runs after formatting.
  Bert: Were asking if we in the WG were thinking of adopting things
        from XSL, since it can do that.
  Bert: Antenna House can do that, Prince cannot.
  Bert: So interesting to work on -- good indexes is a challenge.
  Florian: I think priorities are different when you're doing
           interactive docs.
  Florian: But even then, it's desired.
  Florian: Should have in general.
  SteveZ: Part of secret is doing indexing after you've done the
          document.
  SteveZ: In a batch process you can stick it back up front.
  SteveZ: In an interactive world, there'd be no particular harm in
          being just another page that you accessed.
  SteveZ: Does make sense in an interactive context.
  SteveZ: Stuff in XSL is not that sophisticated that it would be a
          performance problem.

  Bert: Interactive documents.
  Bert: Forgot name of program, but it allows you to see hypertext
        documents.
  Bert: Gives previews of documents being linked.
  Bert: Because screens are big enough now, can have a side panel.
  Bert: Don't have to click the document to see what it is.
  Bert: Maybe it's a nice feature for browsers.
  Bert: Other things that could be done interactively that could
        make reading easier.
  Bert: Exploring a document in various ways.
  Bert: Particularly if you have tables or math, can be very
        interactive.
  Bert: Think of what you could do with data tables.
  Bert: Sort tables, make a graph out of it, like an excel
        spreadsheet.
  Bert: Author says 'this shows a trend', you can make your own
        graph and come up with your own interpretation.
  Bert: You can interact with math formulas.
  Bert: Could also do things with images: zoom in, enhance contrast,
        etc.
  Bert: That's about it.
  * fantasai would like a link to the table algorithm paper
  <liam> [I sent a draft proposal for index fixup in January; I know
         TabAtkins commented on it. it needs work, but is largely
         based on XSL-FO]

  Bert: http://print-css.rocks/
  Bert: Print CSS site, it's a one person site, review of various
        programs for making print from HTML.
  Bert: E.g. found links to PDF Reactor.
  Bert: They can do things like hanging punctuation.
  Bert: You can make really nice documents with PDF Reactor
  <dauwhe> I'd love for them to submit their tests to our test suite
  * fantasai too ;)

  Bert: That's the end of my notes

  glazou: First, thank you Bert for the report, very interesting.
  glazou: What can we take back to the CSSWG, and what is
          interesting for us to discuss, and how?
  glazou: And will the people you talked with help us to standardize?
  Bert: People from Monash, no, their job is teaching and writing
        papers. But you can talk to them and ask for algorithms.
  Bert: They looked at e.g. are these algorithms practical enough to
        use in real time, and in their opinion, yes, they are.
  Bert: YesLogic's Michael Day has more concrete requests.
  Bert: Would be best for him to join the WG, been asking for a long
        time.
  Bert: But it's a very small company, they don't have a lot of
        money, not even an office.
  Bert: They may have half of the market, but it's a small company.
  Bert: We can talk to them. Peter Moulder is active on ML, Michael
        Day not so much, but happy to answer questions.
  Bert: He promised me he'd give me more complaints.
  Bert: So I'll be asking for more complaints.
  Bert: Concrete things they wanted, I mentioned indexes,
  Bert: Michael Day was interested in more constraint-like layouts.
  Bert: A way to express two things side by side, third thing below.
  fantasai: You can do that with Grid
  Bert: I'll think a bit more.

  glazou: To enhance layout, giving constraints between elements.
  glazou: I had a proposal for that.
  glazou: I think Tab also had one.
  glazou: We did some constraint-based layout long ago in Grif.
  glazou: Maybe people at Amaya have some hints.
  Bert: Nobody working on it anymore, but people who worked on it
        might know.
  dauwhe: I also have a good working relationship with Michael Day,
          spend a lot of time on their support forums.
  dauwhe: Do see a lot of pain points there, can try bringing those
          to this group.
  dauwhe: Our system is based on Prince, so I have a lot of
          experience with them.

  skk: As far as I understand, the constraints is coming from users.
  Bert: Constraint is given by author or designer of document.
  skk: Not user.
  Bert: That's an interesting idea. Didn't think about allowing
        reader of document to give constraints.
  skk: If we can think about user also giving constraint to
       document, I think it is interesting.
  skk: At that time we think about personalization of document.
  skk: There is one document, many readers.
  skk: E.g. I like hanging punctuation, but he does not, then user
       can see what they like.
  skk: Personalization can be done using these constraints method.
  Bert: I like that, should think about that.
  skk: Can you let me know later, the ACM paper you mentioned in the
       talk?

  Rossen: In context of all of this, would be great to ask if they
          can look at our Grid proposal.
  Rossen: That's a module where a lot of these constraints can be
          handled very elegantly on the style layer.
  Rossen: Worth having people with that kind of problem space
          looking to the spec at least.
  Rossen: Getting feedback.
  Bert: Michael Day liked my grid better, but he'd looked into the
        new one.
  fantasai: On topic of Grid, planning to transition to CR 2 weeks
            from now, so would be great to get feedback within this
            time, otherwise hard to process.

CSS Text
--------

  Bert: Digipub wanted to talk about character alignment.
  dauwhe: Top of our wish list is aligning on a character in a table
          column.
  dauwhe: This is now in CSS Text 4.
  dauwhe: At TPAC dbaron went and added a bunch of issues about nuts
          and bolts how this would work.
  <dbaron> I added some issues to
https://drafts.csswg.org/css-text-4/#character-alignment
           pointing out things that need to be decided (some are
           technical, but some are actually questions about what the
           users want)

  dauwhe: First issue was 18: how would you align the alignment
          characters themselves if in different font?
  dauwhe: The most common use case is bold row for totals.
  dauwhe: Did a bit of research looking into what existing tools do
          in this situation.
  dauwhe: As far as I can tell, InDesign aligns the origins of the
          glyphs.
  dauwhe: Some discussion on ML whether this is appropriate or
          whether the alignment chars should be centered.
  <dauwhe> https://lists.w3.org/Archives/Public/www-archive/2016Feb/0000.html
  dauwhe: I just put together a diagram.
  dauwhe: This overlays bold/roman text in the same font.
  dauwhe: This is a very small effect.
  <Rossen> https://lists.w3.org/Archives/Public/www-style/2016Jan/0233.html
  * fantasai prefers center alignment
  Florian: I don't have as many examples as Dave has.
  Florian: I've looked into it, talked to others.
  Florian: General agreement that main use case is bold vs regular.
  Florian: Essentially they treat this as an edge case rather than a
           use case.
  Florian: Author should have taken fonts the same size.
  Florian: Not in that situation it's error handling.
  Florian: So they don't have a strong request one way or another.
  Florian: Tiny differences, maybe it doesn't really matter because
           barely visible.
  Florian: If not tiny differences, considered authoring error.

  SteveZ: I had a question about alignment when there is no
          alignment char in the object.
  SteveZ: Numeric value without decimal, e.g.
  SteveZ: What happens if no character?
  dauwhe: I think it inserts the character invisibly after the text.
  dauwhe: e.g. 1000 without .0 at the end.
  dauwhe: If you put other text in there, that would be a problem.
  SteveZ: In error-handling world, need to say what to do.

  franremy: I didn't think about this...
  franremy: Implementations looked into this?
  <franremy> The question was about implementer feedback on the
             proposal.

  fantasai: I think we should center it, it's not biased towards one
            directionality or alignment.
  fantasai: Does anyone have a different opinion?
  <astearns> as long as the virtual character is added in the case
             where it's missing, I'm for center.
  Florian: I don't think centering is the best option. I think it
           could be preferable to align to the left side of the
           character.
  Florian: Very often you are aligning numbers.
  Florian: If you misaligning the right is worse than misaligning
           the left.
  fantasai: I don't agree with you at all.
  Florian: Another way to deal is letter spacing.
  fantasai: No.

  <liam> [ front-alignment (i.e. rtl-dependent) will give the
         strongest result in most cases ]
  <liam> [ i.e. use the text direction of the table, not the actual
         text inside, but align everything to that, because the eye
         sees aligned things more strongly than centred things ]
  <astearns> liam - do you have examples and/or data to back up that
             assertion?
  <liam> i could probably find papers on gestalt perception and
         graphic design; the point is that it's a strong vertical
         axis, weakened when things don't quite line up on it
  <astearns> thanks

  Rossen: Going through 9.1 of text alignment.
  Rossen: As I understand current proposal, happens across table
          cells in same column.
  Rossen: It's a very simplified example.
  Rossen: What about borders, breaking, etc.?
  Florian: This is for the simplest use case.
  Florian: What would you do in such and such case? They say "you
           don't do it in that case."
  Florian: Nobody has any expectations for the complicated cases,
           because for people who need this nobody encounters those
           cases.

  Florian: There is a very strong demand for this from many places.
  Florian: Maybe less on purely web-oriented people, but as soon as
           you hit publishing.
  SteveZ: Financial publishing.
  <astearns> +1 to strong demand
  <liam> [ this isn't an edge case if you have a total of a sum of
         figures and put the total in bold ]
  Florian: Lots of people with tables with numbers in them.

  gregwhitworth: While there's a very specific use case,
  gregwhitworth: Still need to approach the generic standpoint,
                 because people will put anything in a web page.
  Rossen: If you think about the way shrink-to-fit will be affected
          by this... just don't even want to think about it.
  Rossen: Currently in table algorithm, no dependency of content of
          tables besides knowing min/max.

  dbaron: To respond to what Rossen said, look at issues I added to
          the spec
  dbaron: Covered by issues 22-23.
  dbaron: I think it's a solve-able problem, but requires carrying
          around a bit more data.
  dbaron: But also some questions where the users need to say what
          they want before we can define the spec.
  <dbaron> (Issue 22 being the one starting with "This should have a
           formal definition")
  SteveZ: Seems like this is analogous to baseline alignment, just
          in the other dimension.
  SteveZ: Baseline can be arbitrarily aligned as well, yes.
  fantasai: Yes, you have this same problem in the other axis with
            baseline alignment, or in the column axis if the table
            cells' content is orthogonal.
  dbaron: We don't support that.
  Florian: My point is, a lot of these questions, we can't ask for
           examples because there is no such usage. We have to
           decide something, but it can't be based on convention
           because there is no such cases conventionally.

  dauwhe: I've been searching for pathological examples of this.
  dauwhe: So far very little luck.
  dauwhe: I think this may be as publishers, we're operating in
          constrained space.
  dauwhe: Won't see the kind of craziness in web implementations.
  dauwhe: Want to emphasize this is really a problem worth solving.
          Really clear use case.
  dauwhe: I think we should try.
  <Florian> +1 Yes, very clear demand.

  SteveZ: So off topic, or should we do this at all rather than
          where to align the character?
  SteveZ: Can we resolve on where to align the character, move on to
          other things?
  fantasai: I'm advocating center alignment, does anyone object?
  SteveZ: You'd have to compute the center, wouldn't require any
          kind of computation.
  astearns: Liam is advocating for start.
  Florian: Should be left, not start, because even in RTL numbers
           are LTR.
  Florian: Also, if you do have mixed directions across cells, the
           alignment needs to be consistent.
  Florian: Imho center is acceptable, left is preferable.
  astearns: Liam says we should use the writing mode of the table.
  Florian: No, because numbers are LTR.
  Rossen: The vertical centering is not a close behavior to how
          baseline alignment works. Table algorithms are based on
          column sizing as input and rows as results. Restarting row
          layout doesn't have global effect while column does.
  fantasai: If the numbers are shifted, that shift is going to
            affect alignment on both sides, and the shift in the
            number is going to be much bigger.
  fantasai: The right edge has the stronger alignment axis in
            general, so why would you align to the left?
            For example, money has a fixed number of digits on the
            right.
  SteveZ: I disagree, what's to the right is less important than
          what's on the left.
  SteveZ: Dollars more important than cents.
  fantasai: But the left side is not aligned anyway in many cases,
            so why are you caring about alignment on that side?
  Florian: The left edge is the easiest to implement, might as well
           go with that.
  <gregwhitworth> why can't this be controllable by authors, I can
                  buy args for both - even though I think this will
                  be quite complex to impl for a small usecase.
  <fantasai> not worth the switch
  <liam> [ Florian, in the case of LTR numbers in RTL it makes no
         difference; in the case of another character it might ]

  [A straw poll will be taken. Options:
      1. Center
      2. Left ]
  <fantasai> 1
  <zcorpan> abstain
  <Florian> 2, 1
  <SteveZ> left
  <dauwhe> 2 1
  <BradK> 2
  <astearns> 1 2, don't really care which
  <Rossen> abstain
  <liam> 2 out of these
  <gregwhitworth> abstain
  <dbaron> abstain, but maybe slight preference for left
  <koji> abstain
  <skk> abstain
  <Bert> slightly prefer 2 over 1
  <jihye> like 2 more than 1
  <eae> abstain
  <birtles> abstain
  <dbaron> (I'm also fine with table-start)
  <franremy> yes, abstaining

  fantasai: Visually the strongest alignment is going to be on the
            right edge, if you have dollars and cents, hours and
            minutes, etc. So if alignment is what you care about,
            you want the right edge, not the left edge.
  fantasai: So I think you're all wrong, but if you insist I'll put
            it in the spec.

  RESOLVED: left-align the alignment characters

  dauwhe: dpub will do what they can with the other issues, but some
          are beyond our capabilities, as they're implementation
          issues.
  dauwhe: Happy to answer questions about what's happening out in
          the world.

Received on Wednesday, 23 March 2016 21:32:26 UTC