- From: Dael Jackson <daelcss@gmail.com>
- Date: Tue, 24 May 2016 19:35:37 -0400
- To: www-style@w3.org
=========================================
These are the official CSSWG minutes.
Unless you're correcting the minutes,
Please respond by starting a new thread
with an appropriate subject line.
=========================================
CSS Content
-----------
- The conversation started based around an issue that, as written,
cross-references have the potential to create infinite loops.
- plinss suggested solving this using a similar approach to
how footnotes are handled traditionally and Florian agreed
to investigate it.
- The group then moved onto a more philosophical discussion on the
future of the spec. Topics discussed included:
- How can we ensure that browser vendors are okay with the
approach so that they can implement it in the future?
- Is the spec too big to publish?
- How much attention should this spec get as it's not likely
to be prioritized and/or implemented by browsers? Or
should browsers be more interested in this?
- Should epub have a special time on a F2F agenda like the FX
meeting slots?
Testing
-------
- RESOLVED: Drop requirement for author or reviewer metadata
- RESOLVED: Move to primary <link> to spec+section being inferred
from directory structure. Supplemental <link>s must be
inline.
- RESOLVED: spec-shortname/N-levels-of-ignored-subdirectory-
names/frag-id-of-section
- RESOLVED: Remove any title requirement, other than having one
(implied by validity of HTML requirement)
- RESOLVED: testharness.js tests don't need a meta assert (but
reftests still do)
- Moving to Github, move all of our tests to WPT repo, and
stopping future use of Shepherd were discussed, but tabled for
future conversation.
===== FULL MINUTES BELOW ======
Agenda: https://wiki.csswg.org/planning/san-francisco-2016#proposed-agenda-topics
Scribe: fantasai
CSS Content
===========
Page Number References
----------------------
Florian: With regard to cross-references
Florian: It looks like this:
<Florian> a::after { content: target-counter(attr(href, url),
page); }
Florian: Implemented in Antenna House, Prince, and PDFReactor
Florian: Vivliostyle looking into implementing.
Florian: Major issue right now is that as specified, it could be
infinite passes.
Florian: If you have something reference at the very end of 109
Florian: You do re-layout, and then now it's at page 110.
Florian: This goes back to page 109.
Florian: In most cases you do N passes and it eventually stabilizes.
Florian: 20 minutes is okay for print.
Florian: 20 minutes is better than infinite minutes.
Florian: I've never run into infinite loops in the wild, but
multi-passes happens.
Florian: It's already terrible.
Florian: N passes of laying out several hundred pages is a bad idea.
dauwhe: If N is 2 it's not too bad.
<ojan> is there a spec for what we're discussing or just the
mailing list threads?
<fantasai> no spec for this issue
<dauwhe> https://drafts.csswg.org/css-content/#cross-references
plinss: It's only infinite if you allow the race condition to
continue.
plinss: There's other fun things in paged layout that can create
race conditions.
plinss: Can apply a generic solution, detect if you are racing and
stop racing.
Florian: Detecting a loop and stopping is one way out of the
problem.
plinss: Do layout, then check if it moved, then it stays there,
don't bring it back.
Florian: Problem is you end up with incorrect table of contents
numbers.
plinss: No, do second page layout. If it ends up on 110, you treat
is as a widow.
plinss: Leave it out at 110.
plinss: Don't re-layout.
dbaron: You do one pass, do references, say this has to be on at
least this page
dbaron: And shift things to later page.
dbaron: But not earlier page.
Florian: With a forced break priority.
dbaron: Well, might want to do a priority with some ...
Florian: Alternative solution, similar to manual fix-ups
Florian: On first pass you have a placeholder.
Florian: First do pass with a placeholder.
Florian: Then do layout, fill in the text.
Florian: If it's smaller, do some alignment within the placeholder
space.
Florian: If it's larger, might have ink overflow.
[skepticism of this approach]
...
plinss: This is a generic problem. Could have same problem with
widows and orphans.
plinss: This is a generic problem. Gonna run into the same problem
in all sorts of other ways.
plinss: Think we should have a generic solution.
plinss: Not have an author-specified placeholder.
dbaron: The solution is similar to standard solution to footnotes,
which is if you push ...
plinss: If your footnote grows, starts to push marker off, stop
growing at that point.
plinss: Same principle.
plinss: Just before it would create a race condition, you stop.
Florian: Okay, I hadn't thought about that one.
Florian: We have this on our roadmap, not interested in infinite
loop solution.
Florian: We also think that having CSS properties that are
acceptable for print formatters but not Web, is bad.
Florian: Many passes is not good.
plinss: There are other problems that require multiple passes.
plinss: But most can be done in 2 passes, tops.
Florian: So, way you suggested, no need to change syntax.
Florian: One side advantage of placeholders, if you have long
document and it takes awhile to render, have something to
show in the meantime.
dauwhe: Could do that as an author by putting placeholder content
in HTML and then replace with generated content in CSS.
[missed how]
Florian: So most of the time you get 2 passes.
plinss: Most of time you can do in 1 pass.
plinss: Sometimes have to go back and do it again, then stop.
plinss: You might have a rippling effect, but not a looping effect.
plinss: Might have to do nlogn.
ojan: We won't implement any of this, ever.
ojan: It involves laying out the entire document in order to
figure out what page something is on.
ojan: Never doing that.
esprehn: We treat generated content as DOM.
esprehn: So this is going to be an infinite loop for us.
ekimber: Certainly the case that in context of print, potential
for infinite lops is unavoidable.
ekimber: We might not be doing in this context, but then maybe you
move stuff to a different page cuz not enough space, but
that creates enough space, so move it back.
ekimber: So only solution is to have a "stop trying" solution.
esprehn: In our case, at a fundamental level layout can't affect
DOM.
esprehn: You can't say "do layout, and then set attribute based on
layout"
plinss: Layout should never affect DOM. But you don't have to
implement this as modifying the DOM.
...
plinss: It's not DOM tree, it's layout tree.
plinss: You have some situations, but you deal with it.
plinss: But this is implementation details.
myles: We're all agreeing here.
plinss: Nobody is saying that layout affects DOM. I'm saying you
don't have to implement generated content by affecting the
DOM. If you do, can't implement these kinds of features.
gsnedders: If we have 2 implementations, we're fine.
<gsnedders> (sidenote: I was mostly being sarcastic about the fact
that no browser will support it and our 2 impl
requirement, given minutes don't capture tone)
Florian: We have 3.
TabAtkins: How do we deal with things that will not be implemented
by browsers, but are implemented interoperably by other
engines?
TabAtkins: How do we indicate to authors, that this won't work in
browsers?
Florian: Making clear to authors is useful.
Florian: But one goals of what we're doing at Vivliostyle is to
explicitly stop having CSS for Web and CSS for Print
being two different things.
Florian: Want to be integrated.
Florian: We know browsers won't prioritize these features, but
want to make sure they are implementable if they become
prioritized.
...
hober: Florian wants the solution to be something we could
theoretically accept.
tantek: We will never implement vs. this is not a priority, those
are two very different things.
esprehn: They should just do it in Houdini.
Florian: Risking a longer debate.
TabAtkins: Similar to static vs dynamic profile of Selectors.
TabAtkins: We have something that will fundamentally not be done
in CSS processing.
TabAtkins: But works fine in batch processors and JS.
Ebooks vs. Web
--------------
Florian: I'm talking about ebook readers.
Florian: Not batch processors.
TabAtkins: We've already made a somewhat principled split between
main Web stuff for CSS and other stuff.
TabAtkins: Speccing that, and doing it well.
TabAtkins: Fine to do that for ebook or whatever.
TabAtkins: As long as a) it's clear, as in Selectors, that it's
not expected to work in Web pages right now (though
could change in future)
TabAtkins: and also b) scope amount of time we spend on this.
Florian: Jen has some good talks about this, Web design now be
incredibly poor and boring cmp print
Florian: Fantastic design on magazines, not done on Web.
Florian: Web designers should look for inspiration elsewhere than
the Web.
Florian: Maybe pagination isn't part of that.
Florian: Maybe mostly print, but having on the Web might be fine.
esprehn: We should provide primitives, so people can make
libraries, and if it's super popular we can backport to
the Web.
esprehn: Core of Web should be small, and libraries should
implement these things.
Florian: That is okay, but saying " this is for print " don't want
to do.
esprehn: Should say "this is for libraries"
ojan: Being a standard and a spec isn't coupled to whether shipped
in Web or print formatters
ojan: Being a spec doesn't say where it gets implemented. That's a
good thing.
ojan: If we treat the whole room as caring about all the issues [??]
plinss: I think I agree with all of you guys.
plinss: What products implement what features? Depends on the
market.
plinss: Doesn't mean we can't standardize how these things should
behave.
plinss: Whether implemented in browser vendors or libraries or
whatever.
ojan: Why have browsers in the room?
astearns: Because the CSS expertise is in this room.
astearns: We want to avoid the situation like EPUB 3 where they
made up stuff for CSS that was badly designed.
Florian: Also obvious that browser vendors get priority on topics,
Florian: Then again, a whole bunch of people here who are not
browser vendors.
gsnedders: Also worthwhile to point out that a lot of people in
this room have a good idea what can be performantly
implemented within the larger CSS model
gsnedders: regardless of whether implemented in browser or not.
gsnedders: And that's a good thing.
jensimmons: It feels like some of this also is about giant billion
dollar companies saying "this isn't in our interest,
therefore shouldn't be in wg"
jensimmons: But there are smaller vendors, smaller groups that
have other interests
jensimmons: I don't think it's fair to say that if Google will
never implement, we shouldn't spend time on it.
jensimmons: There are some things that are super worth our time to
make sure specced well.
jensimmons: Things that epub and print needs, and a lot of other
things.
jensimmons: Tension between reality of who pays us and let's make
it the Web. Web is cool.
dauwhe: I would just drawing back and looking at question in hand,
this spec is about generated content in general.
dauwhe: Talking about the value of one particular type of counter.
But this applies to a lot of different counters, and there
are a lot of web stuff in this spec.
dauwhe: We reach some resolution on the particular issue, might
not end to specify width for placeholder for page counters
in this circumstance.
dauwhe: As we gain implementation experience we can perhaps
revisit the issue.
Florian: plinss suggestion is maybe a more interesting solution to
explore.
Florian: Given obvious priorities of browsers I think it's fair to
timebox this type of non-browser discussion.
Florian: But definitely don't want it to be on a separate track.
That has been a disaster.
Florian: When liam came to us to say that "I'm sorry to shut down
XSL:FO, but would be ok if CSSWG takes on our use cases"
ojan: Don't have a problem with CSSWG taking on these specs
ojan: Do have a problem with having this whole room to discuss
spec where only 5 people are talking.
ojan: We need to fix that.
dauwhe: General problem.
tantek: That's a scheduling problem.
Florian: Also, Tab and fantasai are always in the discussion.
ojan: I would feel very differently if this was a javascript
library.
ojan: Forcing whole page to re-layout would be fine for a JS
library.
ojan: Fine feature, serves user purpose
Florian: Our UA is a giant JavaScript library.
ojan: Understand about tracks... but how about splitting such
things off into a separate day... houdini ...
astearns: Houdini topics [...]
Florian: Some are native implementations, not JS.
esprehn: What about the rest of the spec?
esprehn: This spec contains a bunch of stuff. Datetime and
document-url, leaders, etc.
esprehn: From our perspective belongs in a library
esprehn: Think it should be in a JS library
esprehn: Be really nice to publish a spec that describes what
browsers really do.
fantasai: That was 2.1.
esprehn: But it didn't.
fantasai: What's missing?
Florian: I put this one on the agenda because we want to
implement, but not how it's been implemented by the other
print formatters.
dauwhe: I came into the WG just as Håkon was leaving, and took
over editorship of GCPM.
dauwhe: Much of it was a .. junkyard.
dauwhe: Stuff just collected in GCPM.
dauwhe: Over past few years, resolutions to move things into their
respective specs.
dauwhe: Moved GC features into GC.
dauwhe: Also now a spec hadn't been published in 13 years, full of
cool and interesting ideas from Håkon and Ian Hickson.
dauwhe: Many things important to the Web, would like to publish.
dauwhe: What we have now is an early-stage ED.
dauwhe: fantasai and I did some work on this to bring it to state
where we can present to WG and publish a new WD.
dauwhe: Beyond that it really is WG decides.
tantek: I'm agreeing what Elliott is saying, if we want to publish
the WD, need to shrink it.
tantek: If we want to publish REC track document, need to have a
concerted effort to split the features that are being
actively developed vs. junkyard features into next level.
fantasai: This *is* a trimmed down spec, it's just not trimmed
down to "browsers only" set of features.
fantasai: If the working group is not taking that as being trimmed
down then the WG is not browser only.
fantasai: We're working towards trimming it to get it to REC.
dbaron: We shouldn't look at Ebook things as being separate from
browsers. Ebooks being in a browser is maybe a good thing.
dauwhe: W3C and IDPF are working towards more integration.
skk: We also creating epub viewer, based on Blink, creating page
generation with C++.
skk: I think generated content spec is not for web browsers, but
for Ebook viewers.
skk: But if we have a chance, but we are using Blink so we want to
talk with Google web browser team to generate the page numbers.
skk: Page numbers are async generated.
skk: We already did the implementation now, but the collaboration
API specified between Blink and us would be very useful.
skk: Also Gecko, better if API is specified.
skk: Such kind of discussion is productive between Web and Ebook.
skk: From Ebook viewer implementer perspective.
Alt Text for Generated Content
------------------------------
dauwhe: One thing in the spec that hasn't been implemented is
trying to design a mechanism to make generated content
more accessible.
dauwhe: Which is a critical point.
fantasai: Not about exposing generation content to a11y--that's
already required.
fantasai: but about some generated content is symbols or pictures,
needs different text for speech.
dauwhe: Our idea was that the content property takes this endless
trying of various bits of strings or replaced elements.
dauwhe: We're proposing that at the end you can put a slash and
then alt text.
dauwhe: Potentially empty, if the string is decorative (e.g.
asterisks)
TabAtkins: Previous proposal was an 'alt' property.
fantasai: It was bad because it doesn't cascade together with
'content'
dauwhe: This would also allow us to provide alt text for images
inserted via content.
plinss: Instead of using URL, maybe using image function?
plinss: ... and put alt text inside that function
Ebooks vs web cont.
-------------------
ojan: Ebooks as part of the Web is a good future.
ojan: That's why I think speccing these things is important.
ojan: But think of it similar to SVG-CSS day.
ojan: Where it's a special cross-functional meeting.
ojan: To have an explicit Ebook Track, not necessarily in
parallel, but be explicit about that section of the F2F.
Florian: I think that's fine, a bit concerned about the boundary
being fuzzy.
ojan: I think being explicit about that might change what's
plausible, practical to implement.
ojan: E.g. target-counter() is great as a feature, but not as a
built-in browser feature.
TabAtkins: Could have JS api about getting the counter value.
Florian: I'm happy about being explicit about scheduling this in
sections, less happy about putting it into the spec.
TabAtkins: I don't like specs that web authors look at but don't
know what to use.
Florian: caniuse.com
TabAtkins: I think authors should know whether such features will
ever be useful.
plinss: I don't want to be picky about language, but going to be
picky about your language.
plinss: Have no problem with your fundamental point "this is
what's expected to be in browsers today."
plinss: But I have a problem with saying "this is never expected
to be on the Web ever."
plinss: Saying up front "We don't ever expect this to work in a
browser", that's going to change the future.
plinss: That's a bad way to scope the future.
astearns: Has to be put in language that the current efforts are
going to be in script, but that they may move into the
browsers. And we don't have a good set of terms to talk
about that migration purpose.
Florian: I'm not sure it's bad to get peoples' hopes up.
Florian: If there's a lot of people that think people really want
it, then it pushes forward.
esprehn: GCPM didn't move forward for many years.
Florian: Because it was a lousy spec.
Florian: I don't want to have that again.
Florian: People just didn't pay attention to junky spec "for print"
...
Page Number References cont.
----------------------------
Florian: target-counter(), when it doesn't refer to pages, I don't
see any reason why it wouldn't be useful on the Web.
Florian: Can do "See Figure 17", nothing wrong with having that on
the Web.
Florian: Until you care about pages, might not do page counter
references, though.
TabAtkins: Mark it at-risk
ojan: What if we had two specs. "here's generated content as it is
in browsers today" and "here's generated content for
pagination"
* fantasai thinks we're arguing over this wayyyy more time than
we're spending on the technical work
plinss: If you say never gonna implement this, I think you're
going to be wrong. Can't tell your business interests 5
years from now.
plinss: But also, you're not the only browser.
esprehn: If you have something doing layout totally different, e.g.
mathml
esprehn: We're not going to spec that, you can't do that.
esprehn: I think making the string thing, take the text content
value of a node, it's a cycle in algorithm, violates
fundamental constraints of how the system work.
Florian: I agree with that, and that's why I want this discussed
here so that we spec it in a way that doesn't violate
fundamental constraints.
[repetition of existing points]
fantasai: Also Elliot I think you're missing the point,
target-counter() can be used for other things than page
numbers.
esprehn: We're not going to ever add more features to counters.
esprehn: We want people to write JS for counters.
plinss: CSS doesn't define what's implemented in C++.
esprehn: HTML had a sorting algorithm, was never implemented, and
it was removed, and people can implement it in JS.
tantek: That was also before incubation and stuff.
<leaverou> esprehn: Do you realize that there are tons of CSS
authors that do not *know* JS?
<leaverou> esprehn: you are complicating the web platform for
authors with this kind of thinking.
<shane> leaverou: implemented in JS won't mean unusable by CSS
authors - the whole point of Houdini is to open CSS up to
extensibility
<leaverou> shane: Using libraries has a ton of overhead. First you
need to find the best library, you need to bear the
bandwidth cost, you need to learn its documentation,
often switch to another library halfway through etc etc
<leaverou> shane: Libraries are not the solution for essential
functionality. Library authors are not spec writers and
will often create worse APIs
<TabAtkins> leaverou: Yes, of course we realize that. We also
realize that the stdlib *cannot* be extended
infinitely, nor do we want to forever hijack *every
feature forever* on how quickly browsers can cycle
their implementation.
<leaverou> TabAtkins: Then allow people to conditionally import
modules, but standardize said modules instead of having
them hunt down libraries which could be terrible.
<leaverou> it seems to me that Houdini is used as an excuse to not
implement anything Google devs don't like
fantasai: This is a draft, you can cut this down we get to the
point that browsers need to consider implementing it.
tantek: If that is true then call it something else.
fantasai: We are replacing those drafts with something new, and
we're going for a FPWD from the old scope of that module
and should have the same shortname, etc.
tantek: I want to suggest a path forward, if there's a dispute wrt
implementability, indicate in the draft
tantek: e.g. put a box explaining the implementation experience
tantek: and caveat of whether it's stabilizing, or is very shaky
etc.
tantek: Give authors anyone else reviewing spec a chance of what's
the context of evaluating this feature.
tantek: This feature has some print implementation, but no browser
implementations yet, pls send feedback, at least that's
conveyed to authors.
tantek: Rather than mis-conveying any intent.
tantek: E.g. features that are widely-implemented vs not
implemented at all labeled separately to help with
reviewers.
tantek: So my request is to put per-feature, implementation
experience and suggestions for feedback.
dauwhe: We have a mechanism for doing that already: test results
per browser.
dauwhe: If we extend that a little bit, can see that Prince/AH/etc.
that are passing, that gives you the information you need.
dauwhe: If you see that the tests pass in blink/safari/edge,
useful information too.
astearns: Script that does the annotations could be more verbose.
dauwhe: You can filter on that information, all sorts of things.
This is data we can get.
dauwhe: Might make us write more tests, all sorts of things.
dbaron: plinss said something earlier about not wanting the spec
to say that a spec was intended for print.
dbaron: My general feeling is that we do not write enough of what
our intent was into the specs, and that hurts our ability
communicate with the ppl reading the specs and trying to
understand them.
dbaron: I think we should have more informative text in specs
saying why what's there is there and what it was intended
for and so on.
dbaron: And one of the other worries I have is, if we don't say
that things are intended for print, then essentially
there's an incentive for me to try and make things not be
in the spec because I'm worried about the risk that some
junior developer at Mozilla will try to implement a thing
that they shouldn't spend time on.
dbaron: But they saw it in the spec and thought we should
implement it.
<tantek> exactly the problem
<tantek> or you get some advocacy groups pushing for something
jensimmons: I just wanted to toss in that I think we should be
careful not to assume what a browser is.
jensimmons: Not say "there's real browsers and there's ebook
readers, and there's print, and we don't care about B
and C".
jensimmons: There are lots of experiments happening around what is
a browser.
jensimmons: This is going to come up a lot. What a browser is
is going to change.
* ojan just wants to clarify that noone said "real" browsers or
that we don't care about ebooks
gregwhitworth: Lea commented in IRC.
gregwhitworth: She was saying that there are a lot of CSS devs
that don't know JS, and we're over-complicating
things.
ChrisL: Lea doesn't disagree with Houdini existing. Disagrees that
Houdini should be used as a blunt hammer for everything.
<leaverou> yup
<leaverou> I've seen it used way too much as an excuse to not
implement or spec things that authors need.
TabAtkins: The circular ones should be dropped from expectation
for browser implementations.
<leaverou> TabAtkins: depends on what you define as a browser. Is
a print formatter a browser?
<TabAtkins> For the purpose of this discussion, "a browser" is
"the things developed by the browser vendors here in
the room".
fantasai: I'm happy to keep discussing technical issues, but I
would like to close the scoping issue.
Florian: Other topic, not GC, is interaction between Media Queries
and @page { size }
Florian: Anyone want to discuss this now?
Florian: Probably should talk about something other than pages,
because everyone is hating on pages right now.
dauwhe: 3D transforms? :)
Testing
=======
gsnedders: We basically have consensus on getting rid of a lot
flags.
<gsnedders> http://testthewebforward.org/docs/css-metadata.html
* fantasai wants a WG resolution on test metadata so to move
forward with conversions
gsnedders: If we want to get browsers actually running the test
suite, then we need to get browser vendors to actually
talk about this.
gsnedders: The metadata thing is actually smaller bit.
gsnedders: Big thing I want to discuss is stuff like how we want
to deal with test review in the future.
gsnedders: How we want to deal with issues on tests in the future.
gsnedders: Because at the moment we have this weird split system
with pull requests on GitHub which get reviewed before
merging.
gsnedders: Whereas other things are just pushed directly to
Mercurial.
gsnedders: In principle, sometime in the future, but realistically
never, they get reviewed.
gsnedders: In my opinion it makes sens to move everything to
github PRs with review before merging.
gsnedders: Realistically, there's no motivating factor for people
to review tests before they're landed.
gsnedders: We saw this with 2.1 as well.
gsnedders: Just moved everything to approved, thousands of tests
never gonna review.
gsnedders: Would like to avoid ending up in that state again.
<tantek> first piece of feedback, no content in HTML/SGML comments
please. e.g. "<!-- YYYY-MM-DD -->" bad. Instead, slap it
on the end of the title attribute, e.g.
title="NAME_OF_REVIEWER, YYYY-MM-DD"
<tantek> (kind of like how people sign documents)
<tantek> could http://testthewebforward.org/docs/css-metadata.html
cite the previous documentation of the CSS WG test suite
documentation?
<tantek> what happened to atomic vs basic tests as Hixie and I had
distinguished so you could check whether something
implemented anything at all?
* tantek digs up https://www.w3.org/Style/CSS/Test/testsuitedocumentation.html
<gsnedders> tantek: we badly need to sort out the fact we have
contradictory documentation all over the place…
<tantek> gsnedders: could you start by citing the previous
documentation?
<tantek> (I mean inline in the css-metadata doc)
<tantek> ah this is the one I was looking for
https://www.w3.org/Style/CSS/Test/guidelines.html is a
good predecessor to cite for css-metadata.html
<tantek> e.g. that guidelines.html has the rel=author, and
rel=help aspects
astearns: Got a bit of history because we had tons of tests
waiting for review in Shepherd, that we new would never
get fixed.
astearns: Tests hanging as Github PRs isn't great either.
dbaron: I don't think reviewing tests is particularly a good use
of time. Particularly the way they have been reviewed so
far.
dbaron: Certain test reviewers have given feedback that ends up
more or less meaning "go away".
dbaron: Also best way to to review tests is to write an
implementation, and to see if the tests pass.
dbaron: And the important thing to check for is that the test
tests what it thinks it's testing.
fantasai: Three things to check:
1) Test pass when it's supposed to pass
2) Test fails when it's supposed to fail
3) Tests what it thinks its testing
Scribe: iank
fantasai: I've seen some tests which only pass sometimes, for
example when it depends on the width of the window.
dbaron: Implementers are probably going to catch #1.
fantasai: You can have tests which are mis-labeled.
fantasai: I think that dbaron has a good point that (1) is going
to be checked by browsers, (2) & (3) you should check
for, you may as well check for everything.
dbaron: I think that web platform tests have worked out a model
that works here.
dbaron: I'm pretty close to telling people to create a folder in
the wpt repo at the moment. This is largely due to
preprocessing issues.
fantasai: In terms of review process to we review then land or
visa versa?
fantasai: Agree that this is an issue we should fix, but back to
gsnedders issue on review.
dbaron: For wpt as a Gecko dev, I can just check in the repo, and
a Mozilla dev will deal with it working.
<gsnedders> in w-p-t there's an extra motive to review: you start
running them when they land
<gsnedders> in csswg-test, there isn't that if the test is already
in the repo
ojan: We have chrome folks that are working on better wpt
integration tests.
zcorpan: WPT, review from the browser vendor means you just merge
the tests.
ojan: Doing future things in wpt will be good for us.
<gsnedders> for w-p-t, the browser vendor review must be public
for free landing.
Florian: Simply the review process, decrease the amount of
meta-data, remove the build system, and once we've done
that we've done a lot. If we do this, then we can
re-evaluate if we need to do other things. I would not
want to start by dropping things that we know that have
value.
ChrisL: Should keep the metadata, as difficult for people to know
what the test is testing.
Scribe: fantasai
gsnedders: WPT has loose rule that the reviewer must be able to
understand what the test is doing.
gsnedders: How you communicate that to the reviewer, isn't really
stated.
gsnedders: Can put in an HTML comment.
gsnedders: Could be completely obvious cuz it's a 2-line test
gsnedders: No point in actually stating forms of how it must be.
gsnedders: Merely gate it on the review
gsnedders: but that only works if you do the review before merging.
fantasai: For our purposes, having links to the sections test
helps us a lot.
fantasai: Not just for review, but also to generate test coverage
reports, implementation reports, and to know which tests
need to be updated when the spec changes.
fantasai: So I think having this in a fixed format is useful and
important to try to keep.
fantasai: With regards to other metadata,
fantasai: WPT have slightly different format because they use a
lot of JS tests that are combined in one file, so having
HTML comments etc. as documentation makes sense, per-
file data is not going to be as useful.
fantasai: Whereas in reftests, more granular per test.
fantasai: I don't see a problem with pulling out the comment on
the test, having it in a standard format rather than
just in a random HTML comment somewhere.
fantasai: Comment being the documentation of what it's testing.
fantasai: Just as a function has a formalized comment at the top
saying its purpose and its parameters,
fantasai: So that it can be used and understood and reimplemented
if necessary, so should a test.
fantasai: This is independent of clearly-written code with good
comments.
Florian: On the assertion itself, I think that's one of the things
that we may need to come back to later
Florian: if it's getting in the way.
Florian: But I don't think we should do this until we've cleared
out all the other problems
Florian: and have determined that it's still a blocker. Not a good
place to start from.
Florian: FWIW, this is less necessary in JS tests. Because you
have an assertion written as code.
Florian: But in CSS tests, that's not the case.
Florian: We might need to drop that description eventually, but
let's not start here.
Florian: As for what we can do, with regards to links to specs, I
think we can sort of have best of both worlds.
Florian: You store your test in predefined directory structure,
points to the right part of the spec.
Florian: But should you wish to link to extra information to link
to other sections of the spec, then you can add more such
links.
Florian: Not blocking.
Florian: Another bit of metadata is the title;
Florian: HTML requires a <title> so we have to have one, but can
say put anything in it.
Florian: With regards to author/reviewer, leave that to source
control.
Florian: With regards to flags, I think we can get rid of most of
them.
Florian: Make them a niche occurrence.
Florian: Spending something like 10-15 min going through the list
might be useful,
Florian: but should do that.
Florian: I think we do that for title, keep assert meta, get rid
of author meta etc. simplify flags, simplify reviewer
process.
Florian: If we just do these bits then we'll have done a lot.
Florian: Then we can discuss why not.
Florian: But not to do more than that
Florian: yet.
<tantek> I don't think it's useful to brainstorm this realtime here.
<tantek> I'd rather read a proposal, e.g. a delta on what
gsnedders has already put forth.
<gsnedders> tantek: as would I, but there's almost no replies on
the mailing list to much of what I say :(
<tantek> gsnedders, yes, I understand why you brought it to the
f2f agenda, that part makes sense
<tantek> my point is that if someone here wants to take the time
to make counter proposals beyond "here's a simple fix"
(e.g. see my suggestions above ;) ) then they can be
actioned to write up a longer proposal
zcorpan: In WPT, you can put link to the spec using <link> element
zcorpan: But can also infer from directory structure
zcorpan: by spec/id.
fantasai: I'm ok with this, as long as that information is
available and people writing tests for cross-section or
cross-spec interaction are able to add <link>s.
zcorpan: OK with requiring assert for reftests, but less so for
testharness.js
[Florian and fantasai agree]
[discussion of where to track reviewer information]
plinss: I would argue against removing any existing metadata, but
remove requirement to put it in.
<tantek> +1
gsnedders: Question remains, how do we flag that a test has been
reviewed?
plinss: If there is a reviewer flag, shepherd will mark it
reviewed. But otherwise Shepherd will track it.
plinss: Merging GitHub PR also counts as a reviewer.
RESOLVED: Drop requirement for author or reviewer metadata
RESOLVED: Move to primary <link> to spec+section being inferred
from directory structure. Supplemental <link>s must be
inline.
<tantek> not hearing consensus
plinss: Do we have nested sections?
fantasai: No. Just spec/fragID.
fantasai: We move sections around or change their nesting level.
But we keep the fragID stable.
plinss: So I propose we have shortname + leafesectionid and N
levels in between.
plinss: We ignore the levels in between, they can be named anything.
fantasai: I don't think there's a need for intermediary directories...
RESOLVED: spec-shortname/N-levels-of-ignored-subdirectory-
names/frag-id-of-section
RESOLVED: for 2.1 use css2/filename/frag-id-of-section
<dbaron> ?: or for 2.1 we can require help links
(remove previous RESOLVED)
plinss: If spec doesn't match this format (e.g. 2.1 which has
chapters) then have to use <link>
RESOLVED: Remove any title requirement, other than having one
(implied by validity of HTML requirement)
tantek: Want to say, don't keep content in HTML comments.
<tantek> can we get a resolution on not putting content in HTML
comments?
* fantasai didn't understand your comment, tantek
RESOLVED: testharness.js tests don't need a meta assert (but
reftests still do)
<tantek> PROPOSED: Drop any requirements or even suggestion to put
test meta info into HTML comments
<tantek> ^^^ yes this is about reviews
gsnedders: Basic agreement on the flags to keep/drop on ML.
...
gsnedders: For WPT, if there's a public review, you can use that
to upstream it to WPT repo.
gsnedders: I think that only affects Edge now :)
Florian: I'm in favor of that, one thing that we're losing if we
move away from Shepherd way of doing things.
Florian: If you are using Shepherd to review, you have access to a
view of tests,
Florian: You don't have that view in GitHub,
Florian: We're assuming people will be diligent and look at the
test on their own machine.
....
fantasai: I think the biggest thing is being able to understand
the test when reviewing it.
fantasai: For me at least, I understand better when I can see the
output and try to connect that to the code I'm looking at.
fantasai: So it's much harder for me to review tests from a GH PR
than to pull down the repo and look at the test locally.
gsnedders: For WPT, PRs by some people ...
zcorpan: I'm on the whitelist, so if I make PR, my PR gets
mirrored on w3ctest.org
zcorpan: In a subdirectory, where I can run it.
zcorpan: Right now there's no automatic link in the PR itself, but
we could add that.
zcorpan: So could add a link directly.
zcorpan: I agree it's useful to be able to easily run the tests
without having to checkout.
Florian: So you can also whitelist PRs from other people.
zcorpan: The whitelist is there to avoid security problem
astearns: So it wouldn't be a problem to take anyone that's
interested in reviewing tests on the whitelist.
Florian: I think having a system like that is valuable, not sure
about blocking on it.
Florian: gtalbot feels pretty strongly against a system which
doesn't have that (ability to easily run the tests).
gsnedders: I think that's fine, practically already have it for
WPT, should make it explicit.
zcorpan: Should make it easier to get there, from the PR,
zcorpan: e.g. have a bot add a link.
Florian: Are we resolving to move to GitHub, review then merge,
and also to merge the system?
Florian: Or is the system a dependency on moving to GitHub?
gsnedders: Should be easy to set up as long as don't need to build.
fantasai: Shouldn't need to build except in [weird very rare cases].
astearns: So seem to want to move to GitHub.
plinss: Suggestion to move all of our tests to WPT repo.
gsnedders: That's a long-term goal shared by browser vendors
(except Microsoft hasn't said anything).
Florian: Short term or long term goal?
plinss: Long term goal since beginning of WPT.
plinss: Block has been our dependency on build system.
plinss: If we eliminate that, then no reason not to move to other
repo.
plinss: I'm happy doing that. Takes Shepherd out of the feature. I
can maintain the historical data, but won't run against
WPT for a long time if ever.
plinss: Would need to keep build system and parts that help us
generate implementation reports.
[Meeting closed.]
[discussion deferred to later]
Received on Tuesday, 24 May 2016 23:36:39 UTC