W3C home > Mailing lists > Public > www-style@w3.org > May 2016

[CSSWG] Minutes San Francisco F2F 2016-05-09 Part IV: CSS Content, Testing [css-content]

From: Dael Jackson <daelcss@gmail.com>
Date: Tue, 24 May 2016 19:35:37 -0400
Message-ID: <CADhPm3tQ1uVm9hdGonVY26GtGxjPnmPof4Wyk6a=PVFR2jz2XQ@mail.gmail.com>
To: www-style@w3.org
=========================================
  These are the official CSSWG minutes.
  Unless you're correcting the minutes,
 Please respond by starting a new thread
   with an appropriate subject line.
=========================================


CSS Content
-----------

  - The conversation started based around an issue that, as written,
      cross-references have the potential to create infinite loops.
      - plinss suggested solving this using a similar approach to
          how footnotes are handled traditionally and Florian agreed
          to investigate it.
  - The group then moved onto a more philosophical discussion on the
      future of the spec. Topics discussed included:
      - How can we ensure that browser vendors are okay with the
          approach so that they can implement it in the future?
      - Is the spec too big to publish?
      - How much attention should this spec get as it's not likely
          to be prioritized and/or implemented by browsers? Or
          should browsers be more interested in this?
      - Should epub have a special time on a F2F agenda like the FX
          meeting slots?

Testing
-------

  - RESOLVED: Drop requirement for author or reviewer metadata
  - RESOLVED: Move to primary <link> to spec+section being inferred
              from directory structure. Supplemental <link>s must be
              inline.
  - RESOLVED: spec-shortname/N-levels-of-ignored-subdirectory-
              names/frag-id-of-section
  - RESOLVED: Remove any title requirement, other than having one
              (implied by validity of HTML requirement)
  - RESOLVED: testharness.js tests don't need a meta assert (but
              reftests still do)
  - Moving to Github, move all of our tests to WPT repo, and
      stopping future use of Shepherd were discussed, but tabled for
      future conversation.

===== FULL MINUTES BELOW ======

Agenda: https://wiki.csswg.org/planning/san-francisco-2016#proposed-agenda-topics

Scribe: fantasai

CSS Content
===========

Page Number References
----------------------

  Florian: With regard to cross-references
  Florian: It looks like this:
  <Florian> a::after { content: target-counter(attr(href, url),
            page); }
  Florian: Implemented in Antenna House, Prince, and PDFReactor
  Florian: Vivliostyle looking into implementing.
  Florian: Major issue right now is that as specified, it could be
           infinite passes.
  Florian: If you have something reference at the very end of 109
  Florian: You do re-layout, and then now it's at page 110.
  Florian: This goes back to page 109.
  Florian: In most cases you do N passes and it eventually stabilizes.
  Florian: 20 minutes is okay for print.
  Florian: 20 minutes is better than infinite minutes.
  Florian: I've never run into infinite loops in the wild, but
           multi-passes happens.
  Florian: It's already terrible.
  Florian: N passes of laying out several hundred pages is a bad idea.
  dauwhe: If N is 2 it's not too bad.

  <ojan> is there a spec for what we're discussing or just the
         mailing list threads?
  <fantasai> no spec for this issue
  <dauwhe> https://drafts.csswg.org/css-content/#cross-references

  plinss: It's only infinite if you allow the race condition to
          continue.
  plinss: There's other fun things in paged layout that can create
          race conditions.
  plinss: Can apply a generic solution, detect if you are racing and
          stop racing.
  Florian: Detecting a loop and stopping is one way out of the
           problem.
  plinss: Do layout, then check if it moved, then it stays there,
          don't bring it back.
  Florian: Problem is you end up with incorrect table of contents
           numbers.
  plinss: No, do second page layout. If it ends up on 110, you treat
          is as a widow.
  plinss: Leave it out at 110.
  plinss: Don't re-layout.
  dbaron: You do one pass, do references, say this has to be on at
          least this page
  dbaron: And shift things to later page.
  dbaron: But not earlier page.
  Florian: With a forced break priority.
  dbaron: Well, might want to do a priority with some ...

  Florian: Alternative solution, similar to manual fix-ups
  Florian: On first pass you have a placeholder.
  Florian: First do pass with a placeholder.
  Florian: Then do layout, fill in the text.
  Florian: If it's smaller, do some alignment within the placeholder
           space.
  Florian: If it's larger, might have ink overflow.
  [skepticism of this approach]
  ...
  plinss: This is a generic problem. Could have same problem with
          widows and orphans.
  plinss: This is a generic problem. Gonna run into the same problem
          in all sorts of other ways.
  plinss: Think we should have a generic solution.
  plinss: Not have an author-specified placeholder.
  dbaron: The solution is similar to standard solution to footnotes,
          which is if you push ...
  plinss: If your footnote grows, starts to push marker off, stop
          growing at that point.
  plinss: Same principle.
  plinss: Just before it would create a race condition, you stop.
  Florian: Okay, I hadn't thought about that one.

  Florian: We have this on our roadmap, not interested in infinite
           loop solution.
  Florian: We also think that having CSS properties that are
           acceptable for print formatters but not Web, is bad.
  Florian: Many passes is not good.
  plinss: There are other problems that require multiple passes.
  plinss: But most can be done in 2 passes, tops.
  Florian: So, way you suggested, no need to change syntax.

  Florian: One side advantage of placeholders, if you have long
           document and it takes awhile to render, have something to
           show in the meantime.
  dauwhe: Could do that as an author by putting placeholder content
          in HTML and then replace with generated content in CSS.
  [missed how]
  Florian: So most of the time you get 2 passes.
  plinss: Most of time you can do in 1 pass.
  plinss: Sometimes have to go back and do it again, then stop.
  plinss: You might have a rippling effect, but not a looping effect.
  plinss: Might have to do nlogn.

  ojan: We won't implement any of this, ever.
  ojan: It involves laying out the entire document in order to
        figure out what page something is on.
  ojan: Never doing that.
  esprehn: We treat generated content as DOM.
  esprehn: So this is going to be an infinite loop for us.
  ekimber: Certainly the case that in context of print, potential
           for infinite lops is unavoidable.
  ekimber: We might not be doing in this context, but then maybe you
           move stuff to a different page cuz not enough space, but
           that creates enough space, so move it back.
  ekimber: So only solution is to have a "stop trying" solution.
  esprehn: In our case, at a fundamental level layout can't affect
           DOM.
  esprehn: You can't say "do layout, and then set attribute based on
           layout"
  plinss: Layout should never affect DOM. But you don't have to
          implement this as modifying the DOM.
  ...
  plinss: It's not DOM tree, it's layout tree.
  plinss: You have some situations, but you deal with it.
  plinss: But this is implementation details.

  myles: We're all agreeing here.
  plinss: Nobody is saying that layout affects DOM. I'm saying you
          don't have to implement generated content by affecting the
          DOM. If you do, can't implement these kinds of features.
  gsnedders: If we have 2 implementations, we're fine.
  <gsnedders> (sidenote: I was mostly being sarcastic about the fact
              that no browser will support it and our 2 impl
              requirement, given minutes don't capture tone)
  Florian: We have 3.

  TabAtkins: How do we deal with things that will not be implemented
             by browsers, but are implemented interoperably by other
             engines?
  TabAtkins: How do we indicate to authors, that this won't work in
             browsers?
  Florian: Making clear to authors is useful.
  Florian: But one goals of what we're doing at Vivliostyle is to
           explicitly stop having CSS for Web and CSS for Print
           being two different things.
  Florian: Want to be integrated.
  Florian: We know browsers won't prioritize these features, but
           want to make sure they are implementable if they become
           prioritized.
  ...
  hober: Florian wants the solution to be something we could
         theoretically accept.
  tantek: We will never implement vs. this is not a priority, those
          are two very different things.
  esprehn: They should just do it in Houdini.
  Florian: Risking a longer debate.
  TabAtkins: Similar to static vs dynamic profile of Selectors.
  TabAtkins: We have something that will fundamentally not be done
             in CSS processing.
  TabAtkins: But works fine in batch processors and JS.

Ebooks vs. Web
--------------

  Florian: I'm talking about ebook readers.
  Florian: Not batch processors.
  TabAtkins: We've already made a somewhat principled split between
             main Web stuff for CSS and other stuff.
  TabAtkins: Speccing that, and doing it well.
  TabAtkins: Fine to do that for ebook or whatever.
  TabAtkins: As long as a) it's clear, as in Selectors, that it's
             not expected to work in Web pages right now (though
             could change in future)
  TabAtkins: and also b) scope amount of time we spend on this.

  Florian: Jen has some good talks about this, Web design now be
           incredibly poor and boring cmp print
  Florian: Fantastic design on magazines, not done on Web.
  Florian: Web designers should look for inspiration elsewhere than
           the Web.
  Florian: Maybe pagination isn't part of that.
  Florian: Maybe mostly print, but having on the Web might be fine.

  esprehn: We should provide primitives, so people can make
           libraries, and if it's super popular we can backport to
           the Web.
  esprehn: Core of Web should be small, and libraries should
           implement these things.
  Florian: That is okay, but saying " this is for print " don't want
           to do.
  esprehn: Should say "this is for libraries"
  ojan: Being a standard and a spec isn't coupled to whether shipped
        in Web or print formatters
  ojan: Being a spec doesn't say where it gets implemented. That's a
        good thing.
  ojan: If we treat the whole room as caring about all the issues [??]

  plinss: I think I agree with all of you guys.
  plinss: What products implement what features? Depends on the
          market.
  plinss: Doesn't mean we can't standardize how these things should
          behave.
  plinss: Whether implemented in browser vendors or libraries or
          whatever.
  ojan: Why have browsers in the room?
  astearns: Because the CSS expertise is in this room.
  astearns: We want to avoid the situation like EPUB 3 where they
            made up stuff for CSS that was badly designed.
  Florian: Also obvious that browser vendors get priority on topics,
  Florian: Then again, a whole bunch of people here who are not
           browser vendors.
  gsnedders: Also worthwhile to point out that a lot of people in
             this room have a good idea what can be performantly
             implemented within the larger CSS model
  gsnedders: regardless of whether implemented in browser or not.
  gsnedders: And that's a good thing.

  jensimmons: It feels like some of this also is about giant billion
              dollar companies saying "this isn't in our interest,
              therefore shouldn't be in wg"
  jensimmons: But there are smaller vendors, smaller groups that
              have other interests
  jensimmons: I don't think it's fair to say that if Google will
              never implement, we shouldn't spend time on it.
  jensimmons: There are some things that are super worth our time to
              make sure specced well.
  jensimmons: Things that epub and print needs, and a lot of other
              things.
  jensimmons: Tension between reality of who pays us and let's make
              it the Web. Web is cool.

  dauwhe: I would just drawing back and looking at question in hand,
          this spec is about generated content in general.
  dauwhe: Talking about the value of one particular type of counter.
          But this applies to a lot of different counters, and there
          are a lot of web stuff in this spec.
  dauwhe: We reach some resolution on the particular issue, might
          not end to specify width for placeholder for page counters
          in this circumstance.
  dauwhe: As we gain implementation experience we can perhaps
          revisit the issue.

  Florian: plinss suggestion is maybe a more interesting solution to
           explore.
  Florian: Given obvious priorities of browsers I think it's fair to
           timebox this type of non-browser discussion.
  Florian: But definitely don't want it to be on a separate track.
           That has been a disaster.
  Florian: When liam came to us to say that "I'm sorry to shut down
           XSL:FO, but would be ok if CSSWG takes on our use cases"
  ojan: Don't have a problem with CSSWG taking on these specs
  ojan: Do have a problem with having this whole room to discuss
        spec where only 5 people are talking.
  ojan: We need to fix that.
  dauwhe: General problem.
  tantek: That's a scheduling problem.
  Florian: Also, Tab and fantasai are always in the discussion.

  ojan: I would feel very differently if this was a javascript
        library.
  ojan: Forcing whole page to re-layout would be fine for a JS
        library.
  ojan: Fine feature, serves user purpose
  Florian: Our UA is a giant JavaScript library.
  ojan: Understand about tracks... but how about splitting such
        things off into a separate day... houdini ...
  astearns: Houdini topics [...]
  Florian: Some are native implementations, not JS.

  esprehn: What about the rest of the spec?
  esprehn: This spec contains a bunch of stuff. Datetime and
           document-url, leaders, etc.
  esprehn: From our perspective belongs in a library
  esprehn: Think it should be in a JS library
  esprehn: Be really nice to publish a spec that describes what
           browsers really do.
  fantasai: That was 2.1.
  esprehn: But it didn't.
  fantasai: What's missing?
  Florian: I put this one on the agenda because we want to
           implement, but not how it's been implemented by the other
           print formatters.

  dauwhe: I came into the WG just as Håkon was leaving, and took
          over editorship of GCPM.
  dauwhe: Much of it was a .. junkyard.
  dauwhe: Stuff just collected in GCPM.
  dauwhe: Over past few years, resolutions to move things into their
          respective specs.
  dauwhe: Moved GC features into GC.
  dauwhe: Also now a spec hadn't been published in 13 years, full of
          cool and interesting ideas from Håkon and Ian Hickson.
  dauwhe: Many things important to the Web, would like to publish.
  dauwhe: What we have now is an early-stage ED.
  dauwhe: fantasai and I did some work on this to bring it to state
          where we can present to WG and publish a new WD.
  dauwhe: Beyond that it really is WG decides.

  tantek: I'm agreeing what Elliott is saying, if we want to publish
          the WD, need to shrink it.
  tantek: If we want to publish REC track document, need to have a
          concerted effort to split the features that are being
          actively developed vs. junkyard features into next level.
  fantasai: This *is* a trimmed down spec, it's just not trimmed
            down to "browsers only" set of features.
  fantasai: If the working group is not taking that as being trimmed
            down then the WG is not browser only.
  fantasai: We're working towards trimming it to get it to REC.
  dbaron: We shouldn't look at Ebook things as being separate from
          browsers. Ebooks being in a browser is maybe a good thing.
  dauwhe: W3C and IDPF are working towards more integration.

  skk: We also creating epub viewer, based on Blink, creating page
       generation with C++.
  skk: I think generated content spec is not for web browsers, but
       for Ebook viewers.
  skk: But if we have a chance, but we are using Blink so we want to
       talk with Google web browser team to generate the page numbers.
  skk: Page numbers are async generated.
  skk: We already did the implementation now, but the collaboration
       API specified between Blink and us would be very useful.
  skk: Also Gecko, better if API is specified.
  skk: Such kind of discussion is productive between Web and Ebook.
  skk: From Ebook viewer implementer perspective.

Alt Text for Generated Content
------------------------------

  dauwhe: One thing in the spec that hasn't been implemented is
          trying to design a mechanism to make generated content
          more accessible.
  dauwhe: Which is a critical point.
  fantasai: Not about exposing generation content to a11y--that's
            already required.
  fantasai: but about some generated content is symbols or pictures,
            needs different text for speech.
  dauwhe: Our idea was that the content property takes this endless
          trying of various bits of strings or replaced elements.
  dauwhe: We're proposing that at the end you can put a slash and
          then alt text.
  dauwhe: Potentially empty, if the string is decorative (e.g.
          asterisks)
  TabAtkins: Previous proposal was an 'alt' property.
  fantasai: It was bad because it doesn't cascade together with
            'content'
  dauwhe: This would also allow us to provide alt text for images
          inserted via content.
  plinss: Instead of using URL, maybe using image function?
  plinss: ... and put alt text inside that function

Ebooks vs web cont.
-------------------

  ojan: Ebooks as part of the Web is a good future.
  ojan: That's why I think speccing these things is important.
  ojan: But think of it similar to SVG-CSS day.
  ojan: Where it's a special cross-functional meeting.
  ojan: To have an explicit Ebook Track, not necessarily in
        parallel, but be explicit about that section of the F2F.
  Florian: I think that's fine, a bit concerned about the boundary
           being fuzzy.
  ojan: I think being explicit about that might change what's
        plausible, practical to implement.
  ojan: E.g. target-counter() is great as a feature, but not as a
        built-in browser feature.
  TabAtkins: Could have JS api about getting the counter value.
  Florian: I'm happy about being explicit about scheduling this in
           sections, less happy about putting it into the spec.
  TabAtkins: I don't like specs that web authors look at but don't
             know what to use.
  Florian: caniuse.com
  TabAtkins: I think authors should know whether such features will
             ever be useful.
  plinss: I don't want to be picky about language, but going to be
          picky about your language.
  plinss: Have no problem with your fundamental point "this is
          what's expected to be in browsers today."
  plinss: But I have a problem with saying "this is never expected
          to be on the Web ever."
  plinss: Saying up front "We don't ever expect this to work in a
          browser", that's going to change the future.
  plinss: That's a bad way to scope the future.

  astearns: Has to be put in language that the current efforts are
            going to be in script, but that they may move into the
            browsers. And we don't have a good set of terms to talk
            about that migration purpose.
  Florian: I'm not sure it's bad to get peoples' hopes up.
  Florian: If there's a lot of people that think people really want
           it, then it pushes forward.
  esprehn: GCPM didn't move forward for many years.
  Florian: Because it was a lousy spec.
  Florian: I don't want to have that again.
  Florian: People just didn't pay attention to junky spec "for print"
  ...

Page Number References cont.
----------------------------

  Florian: target-counter(), when it doesn't refer to pages, I don't
           see any reason why it wouldn't be useful on the Web.
  Florian: Can do "See Figure 17", nothing wrong with having that on
           the Web.
  Florian: Until you care about pages, might not do page counter
           references, though.
  TabAtkins: Mark it at-risk
  ojan: What if we had two specs. "here's generated content as it is
        in browsers today" and "here's generated content for
        pagination"
  * fantasai thinks we're arguing over this wayyyy more time than
             we're spending on the technical work

  plinss: If you say never gonna implement this, I think you're
          going to be wrong. Can't tell your business interests 5
          years from now.
  plinss: But also, you're not the only browser.
  esprehn: If you have something doing layout totally different, e.g.
           mathml
  esprehn: We're not going to spec that, you can't do that.
  esprehn: I think making the string thing, take the text content
           value of a node, it's a cycle in algorithm, violates
           fundamental constraints of how the system work.
  Florian: I agree with that, and that's why I want this discussed
           here so that we spec it in a way that doesn't violate
           fundamental constraints.
  [repetition of existing points]

  fantasai: Also Elliot I think you're missing the point,
            target-counter() can be used for other things than page
            numbers.
  esprehn: We're not going to ever add more features to counters.
  esprehn: We want people to write JS for counters.
  plinss: CSS doesn't define what's implemented in C++.
  esprehn: HTML had a sorting algorithm, was never implemented, and
           it was removed, and people can implement it in JS.
  tantek: That was also before incubation and stuff.

  <leaverou> esprehn: Do you realize that there are tons of CSS
             authors that do not *know* JS?
  <leaverou> esprehn: you are complicating the web platform for
             authors with this kind of thinking.
  <shane> leaverou: implemented in JS won't mean unusable by CSS
          authors - the whole point of Houdini is to open CSS up to
          extensibility
  <leaverou> shane: Using libraries has a ton of overhead. First you
             need to find the best library, you need to bear the
             bandwidth cost, you need to learn its documentation,
             often switch to another library halfway through etc etc
  <leaverou> shane: Libraries are not the solution for essential
             functionality. Library authors are not spec writers and
             will often create worse APIs
  <TabAtkins> leaverou: Yes, of course we realize that. We also
              realize that the stdlib *cannot* be extended
              infinitely, nor do we want to forever hijack *every
              feature forever* on how quickly browsers can cycle
              their implementation.
  <leaverou> TabAtkins: Then allow people to conditionally import
             modules, but standardize said modules instead of having
             them hunt down libraries which could be terrible.
  <leaverou> it seems to me that Houdini is used as an excuse to not
             implement anything Google devs don't like

  fantasai: This is a draft, you can cut this down we get to the
            point that browsers need to consider implementing it.
  tantek: If that is true then call it something else.
  fantasai: We are replacing those drafts with something new, and
            we're going for a FPWD from the old scope of that module
            and should have the same shortname, etc.
  tantek: I want to suggest a path forward, if there's a dispute wrt
          implementability, indicate in the draft
  tantek: e.g. put a box explaining the implementation experience
  tantek: and caveat of whether it's stabilizing, or is very shaky
          etc.
  tantek: Give authors anyone else reviewing spec a chance of what's
          the context of evaluating this feature.
  tantek: This feature has some print implementation, but no browser
          implementations yet, pls send feedback, at least that's
          conveyed to authors.
  tantek: Rather than mis-conveying any intent.
  tantek: E.g. features that are widely-implemented vs not
          implemented at all labeled separately to help with
          reviewers.
  tantek: So my request is to put per-feature, implementation
          experience and suggestions for feedback.
  dauwhe: We have a mechanism for doing that already: test results
          per browser.
  dauwhe: If we extend that a little bit, can see that Prince/AH/etc.
          that are passing, that gives you the information you need.
  dauwhe: If you see that the tests pass in blink/safari/edge,
          useful information too.
  astearns: Script that does the annotations could be more verbose.
  dauwhe: You can filter on that information, all sorts of things.
          This is data we can get.
  dauwhe: Might make us write more tests, all sorts of things.

  dbaron: plinss said something earlier about not wanting the spec
          to say that a spec was intended for print.
  dbaron: My general feeling is that we do not write enough of what
          our intent was into the specs, and that hurts our ability
          communicate with the ppl reading the specs and trying to
          understand them.
  dbaron: I think we should have more informative text in specs
          saying why what's there is there and what it was intended
          for and so on.
  dbaron: And one of the other worries I have is, if we don't say
          that things are intended for print, then essentially
          there's an incentive for me to try and make things not be
          in the spec because I'm worried about the risk that some
          junior developer at Mozilla will try to implement a thing
          that they shouldn't spend time on.
  dbaron: But they saw it in the spec and thought we should
          implement it.
  <tantek> exactly the problem
  <tantek> or you get some advocacy groups pushing for something

  jensimmons: I just wanted to toss in that I think we should be
              careful not to assume what a browser is.
  jensimmons: Not say "there's real browsers and there's ebook
              readers, and there's print, and we don't care about B
              and C".
  jensimmons: There are lots of experiments happening around what is
              a browser.
  jensimmons: This is going to come up a lot. What a browser is
              is going to change.
  * ojan just wants to clarify that noone said "real" browsers or
         that we don't care about ebooks

  gregwhitworth: Lea commented in IRC.
  gregwhitworth: She was saying that there are a lot of CSS devs
                 that don't know JS, and we're over-complicating
                 things.
  ChrisL: Lea doesn't disagree with Houdini existing. Disagrees that
          Houdini should be used as a blunt hammer for everything.
  <leaverou> yup
  <leaverou> I've seen it used way too much as an excuse to not
             implement or spec things that authors need.

  TabAtkins: The circular ones should be dropped from expectation
             for browser implementations.
  <leaverou> TabAtkins: depends on what you define as a browser. Is
             a print formatter a browser?
  <TabAtkins> For the purpose of this discussion, "a browser" is
              "the things developed by the browser vendors here in
              the room".

  fantasai: I'm happy to keep discussing technical issues, but I
            would like to close the scoping issue.

  Florian: Other topic, not GC, is interaction between Media Queries
           and @page { size }
  Florian: Anyone want to discuss this now?
  Florian: Probably should talk about something other than pages,
           because everyone is hating on pages right now.
  dauwhe: 3D transforms? :)

Testing
=======

  gsnedders: We basically have consensus on getting rid of a lot
             flags.
  <gsnedders> http://testthewebforward.org/docs/css-metadata.html
  * fantasai wants a WG resolution on test metadata so to move
             forward with conversions
  gsnedders: If we want to get browsers actually running the test
             suite, then we need to get browser vendors to actually
             talk about this.
  gsnedders: The metadata thing is actually smaller bit.

  gsnedders: Big thing I want to discuss is stuff like how we want
             to deal with test review in the future.
  gsnedders: How we want to deal with issues on tests in the future.
  gsnedders: Because at the moment we have this weird split system
             with pull requests on GitHub which get reviewed before
             merging.
  gsnedders: Whereas other things are just pushed directly to
             Mercurial.
  gsnedders: In principle, sometime in the future, but realistically
             never, they get reviewed.
  gsnedders: In my opinion it makes sens to move everything to
             github PRs with review before merging.
  gsnedders: Realistically, there's no motivating factor for people
             to review tests before they're landed.
  gsnedders: We saw this with 2.1 as well.
  gsnedders: Just moved everything to approved, thousands of tests
             never gonna review.
  gsnedders: Would like to avoid ending up in that state again.

  <tantek> first piece of feedback, no content in HTML/SGML comments
           please. e.g. "<!-- YYYY-MM-DD -->" bad. Instead, slap it
           on the end of the title attribute, e.g.
           title="NAME_OF_REVIEWER, YYYY-MM-DD"
  <tantek> (kind of like how people sign documents)
  <tantek> could http://testthewebforward.org/docs/css-metadata.html
           cite the previous documentation of the CSS WG test suite
           documentation?
  <tantek> what happened to atomic vs basic tests as Hixie and I had
           distinguished so you could check whether something
           implemented anything at all?
  * tantek digs up https://www.w3.org/Style/CSS/Test/testsuitedocumentation.html
  <gsnedders> tantek: we badly need to sort out the fact we have
              contradictory documentation all over the place…
  <tantek> gsnedders: could you start by citing the previous
           documentation?
  <tantek> (I mean inline in the css-metadata doc)
  <tantek> ah this is the one I was looking for
           https://www.w3.org/Style/CSS/Test/guidelines.html is a
           good predecessor to cite for css-metadata.html
  <tantek> e.g. that guidelines.html has the rel=author, and
           rel=help aspects

  astearns: Got a bit of history because we had tons of tests
            waiting for review in Shepherd, that we new would never
            get fixed.
  astearns: Tests hanging as Github PRs isn't great either.
  dbaron: I don't think reviewing tests is particularly a good use
          of time. Particularly the way they have been reviewed so
          far.
  dbaron: Certain test reviewers have given feedback that ends up
          more or less meaning "go away".
  dbaron: Also best way to to review tests is to write an
          implementation, and to see if the tests pass.
  dbaron: And the important thing to check for is that the test
          tests what it thinks it's testing.

  fantasai: Three things to check:
            1) Test pass when it's supposed to pass
            2) Test fails when it's supposed to fail
            3) Tests what it thinks its testing
  Scribe: iank
  fantasai: I've seen some tests which only pass sometimes, for
            example when it depends on the width of the window.
  dbaron: Implementers are probably going to catch #1.
  fantasai: You can have tests which are mis-labeled.
  fantasai: I think that dbaron has a good point that (1) is going
            to be checked by browsers, (2) & (3) you should check
            for, you may as well check for everything.
  dbaron: I think that web platform tests have worked out a model
          that works here.

  dbaron: I'm pretty close to telling people to create a folder in
          the wpt repo at the moment. This is largely due to
          preprocessing issues.
  fantasai: In terms of review process to we review then land or
            visa versa?
  fantasai: Agree that this is an issue we should fix, but back to
            gsnedders issue on review.
  dbaron: For wpt as a Gecko dev, I can just check in the repo, and
          a Mozilla dev will deal with it working.
  <gsnedders> in w-p-t there's an extra motive to review: you start
              running them when they land
  <gsnedders> in csswg-test, there isn't that if the test is already
              in the repo
  ojan: We have chrome folks that are working on better wpt
        integration tests.
  zcorpan: WPT, review from the browser vendor means you just merge
           the tests.
  ojan: Doing future things in wpt will be good for us.
  <gsnedders> for w-p-t, the browser vendor review must be public
              for free landing.

  Florian: Simply the review process, decrease the amount of
           meta-data, remove the build system, and once we've done
           that we've done a lot. If we do this, then we can
           re-evaluate if we need to do other things. I would not
           want to start by dropping things that we know that have
           value.

  ChrisL: Should keep the metadata, as difficult for people to know
          what the test is testing.
Scribe: fantasai
  gsnedders: WPT has loose rule that the reviewer must be able to
             understand what the test is doing.
  gsnedders: How you communicate that to the reviewer, isn't really
             stated.
  gsnedders: Can put in an HTML comment.
  gsnedders: Could be completely obvious cuz it's a 2-line test
  gsnedders: No point in actually stating forms of how it must be.
  gsnedders: Merely gate it on the review
  gsnedders: but that only works if you do the review before merging.

  fantasai: For our purposes, having links to the sections test
            helps us a lot.
  fantasai: Not just for review, but also to generate test coverage
            reports, implementation reports, and to know which tests
            need to be updated when the spec changes.
  fantasai: So I think having this in a fixed format is useful and
            important to try to keep.

  fantasai: With regards to other metadata,
  fantasai: WPT have slightly different format because they use a
            lot of JS tests that are combined in one file, so having
            HTML comments etc. as documentation makes sense, per-
            file data is not going to be as useful.
  fantasai: Whereas in reftests, more granular per test.
  fantasai: I don't see a problem with pulling out the comment on
            the test, having it in a standard format rather than
            just in a random HTML comment somewhere.
  fantasai: Comment being the documentation of what it's testing.
  fantasai: Just as a function has a formalized comment at the top
            saying its purpose and its parameters,
  fantasai: So that it can be used and understood and reimplemented
            if necessary, so should a test.
  fantasai: This is independent of clearly-written code with good
            comments.

  Florian: On the assertion itself, I think that's one of the things
           that we may need to come back to later
  Florian: if it's getting in the way.
  Florian: But I don't think we should do this until we've cleared
           out all the other problems
  Florian: and have determined that it's still a blocker. Not a good
           place to start from.
  Florian: FWIW, this is less necessary in JS tests. Because you
           have an assertion written as code.
  Florian: But in CSS tests, that's not the case.
  Florian: We might need to drop that description eventually, but
           let's not start here.

  Florian: As for what we can do, with regards to links to specs, I
           think we can sort of have best of both worlds.
  Florian: You store your test in predefined directory structure,
           points to the right part of the spec.
  Florian: But should you wish to link to extra information to link
           to other sections of the spec, then you can add more such
           links.
  Florian: Not blocking.

  Florian: Another bit of metadata is the title;
  Florian: HTML requires a <title> so we have to have one, but can
           say put anything in it.
  Florian: With regards to author/reviewer, leave that to source
           control.
  Florian: With regards to flags, I think we can get rid of most of
           them.
  Florian: Make them a niche occurrence.
  Florian: Spending something like 10-15 min going through the list
           might be useful,
  Florian: but should do that.
  Florian: I think we do that for title, keep assert meta, get rid
           of author meta etc. simplify flags, simplify reviewer
           process.
  Florian: If we just do these bits then we'll have done a lot.
  Florian: Then we can discuss why not.
  Florian: But not to do more than that
  Florian: yet.

  <tantek> I don't think it's useful to brainstorm this realtime here.
  <tantek> I'd rather read a proposal, e.g. a delta on what
           gsnedders has already put forth.
  <gsnedders> tantek: as would I, but there's almost no replies on
              the mailing list to much of what I say :(
  <tantek> gsnedders, yes, I understand why you brought it to the
           f2f agenda, that part makes sense
  <tantek> my point is that if someone here wants to take the time
           to make counter proposals beyond "here's a simple fix"
           (e.g. see my suggestions above ;) ) then they can be
           actioned to write up a longer proposal

  zcorpan: In WPT, you can put link to the spec using <link> element
  zcorpan: But can also infer from directory structure
  zcorpan: by spec/id.
  fantasai: I'm ok with this, as long as that information is
            available and people writing tests for cross-section or
            cross-spec interaction are able to add <link>s.
  zcorpan: OK with requiring assert for reftests, but less so for
           testharness.js
  [Florian and fantasai agree]

  [discussion of where to track reviewer information]

  plinss: I would argue against removing any existing metadata, but
          remove requirement to put it in.
  <tantek> +1
  gsnedders: Question remains, how do we flag that a test has been
             reviewed?
  plinss: If there is a reviewer flag, shepherd will mark it
          reviewed. But otherwise Shepherd will track it.
  plinss: Merging GitHub PR also counts as a reviewer.

  RESOLVED: Drop requirement for author or reviewer metadata
  RESOLVED: Move to primary <link> to spec+section being inferred
            from directory structure. Supplemental <link>s must be
            inline.

  <tantek> not hearing consensus
  plinss: Do we have nested sections?
  fantasai: No. Just spec/fragID.
  fantasai: We move sections around or change their nesting level.
            But we keep the fragID stable.
  plinss: So I propose we have shortname + leafesectionid and N
          levels in between.
  plinss: We ignore the levels in between, they can be named anything.
  fantasai: I don't think there's a need for intermediary directories...

  RESOLVED: spec-shortname/N-levels-of-ignored-subdirectory-
            names/frag-id-of-section

  RESOLVED: for 2.1 use css2/filename/frag-id-of-section
  <dbaron> ?: or for 2.1 we can require help links
  (remove previous RESOLVED)
  plinss: If spec doesn't match this format (e.g. 2.1 which has
          chapters) then have to use <link>

  RESOLVED: Remove any title requirement, other than having one
            (implied by validity of HTML requirement)

  tantek: Want to say, don't keep content in HTML comments.
  <tantek> can we get a resolution on not putting content in HTML
           comments?
  * fantasai didn't understand your comment, tantek

  RESOLVED: testharness.js tests don't need a meta assert (but
            reftests still do)

  <tantek> PROPOSED: Drop any requirements or even suggestion to put
           test meta info into HTML comments
  <tantek> ^^^ yes this is about reviews

  gsnedders: Basic agreement on the flags to keep/drop on ML.
  ...
  gsnedders: For WPT, if there's a public review, you can use that
             to upstream it to WPT repo.
  gsnedders: I think that only affects Edge now :)

  Florian: I'm in favor of that, one thing that we're losing if we
           move away from Shepherd way of doing things.
  Florian: If you are using Shepherd to review, you have access to a
           view of tests,
  Florian: You don't have that view in GitHub,
  Florian: We're assuming people will be diligent and look at the
           test on their own machine.
  ....
  fantasai: I think the biggest thing is being able to understand
            the test when reviewing it.
  fantasai: For me at least, I understand better when I can see the
            output and try to connect that to the code I'm looking at.
  fantasai: So it's much harder for me to review tests from a GH PR
            than to pull down the repo and look at the test locally.

  gsnedders: For WPT, PRs by some people ...
  zcorpan: I'm on the whitelist, so if I make PR, my PR gets
           mirrored on w3ctest.org
  zcorpan: In a subdirectory, where I can run it.
  zcorpan: Right now there's no automatic link in the PR itself, but
           we could add that.
  zcorpan: So could add a link directly.
  zcorpan: I agree it's useful to be able to easily run the tests
           without having to checkout.
  Florian: So you can also whitelist PRs from other people.
  zcorpan: The whitelist is there to avoid security problem
  astearns: So it wouldn't be a problem to take anyone that's
            interested in reviewing tests on the whitelist.
  Florian: I think having a system like that is valuable, not sure
           about blocking on it.
  Florian: gtalbot feels pretty strongly against a system which
           doesn't have that (ability to easily run the tests).
  gsnedders: I think that's fine, practically already have it for
             WPT, should make it explicit.
  zcorpan: Should make it easier to get there, from the PR,
  zcorpan: e.g. have a bot add a link.

  Florian: Are we resolving to move to GitHub, review then merge,
           and also to merge the system?
  Florian: Or is the system a dependency on moving to GitHub?
  gsnedders: Should be easy to set up as long as don't need to build.
  fantasai: Shouldn't need to build except in [weird very rare cases].
  astearns: So seem to want to move to GitHub.

  plinss: Suggestion to move all of our tests to WPT repo.
  gsnedders: That's a long-term goal shared by browser vendors
             (except Microsoft hasn't said anything).
  Florian: Short term or long term goal?
  plinss: Long term goal since beginning of WPT.
  plinss: Block has been our dependency on build system.
  plinss: If we eliminate that, then no reason not to move to other
          repo.
  plinss: I'm happy doing that. Takes Shepherd out of the feature. I
          can maintain the historical data, but won't run against
          WPT for a long time if ever.
  plinss: Would need to keep build system and parts that help us
          generate implementation reports.

  [Meeting closed.]
  [discussion deferred to later]
Received on Tuesday, 24 May 2016 23:36:39 UTC

This archive was generated by hypermail 2.3.1 : Tuesday, 24 May 2016 23:36:39 UTC