- From: Geoffrey Sneddon <me@gsnedders.com>
- Date: Mon, 25 Sep 2017 20:14:07 +0900
- To: Philip Jägenstedt <foolip@google.com>
- Cc: Alan Stearns <stearns@adobe.com>, James Graham <james@hoppipolla.co.uk>, public-test-infra <public-test-infra@w3.org>, Florian Rivoal <florian@rivoal.net>, Peter Linss <peter@linss.com>
- Message-ID: <CAHKdfMjQt5ugL9gwbGsVG4ZDaDN83_Aretq8CksP_+jFyEyseQ@mail.gmail.com>
On Mon, Sep 25, 2017 at 7:38 PM, Philip Jägenstedt <foolip@google.com> wrote: > On Mon, Sep 25, 2017 at 12:08 PM Geoffrey Sneddon <me@gsnedders.com> > wrote: > >> >> >> On Mon, Sep 25, 2017 at 6:31 PM, Philip Jägenstedt <foolip@google.com> >> wrote: >> >>> On Sun, Sep 24, 2017 at 3:48 AM Geoffrey Sneddon <me@gsnedders.com> >>> wrote: >>> >>>> On Sun, Sep 24, 2017 at 12:23 AM, Philip Jägenstedt <foolip@google.com> >>>> wrote: >>>> >>>>> On Sat, Sep 23, 2017 at 11:34 PM Geoffrey Sneddon <me@gsnedders.com> >>>>> wrote: >>>>> >>>>>> That said, the problems around the anchors could also be resolved by >>>>>> the CSS WG amending their tooling to put unknown anchors in a new "unknown" >>>>>> section, but I believe this would involve changes both in the build system >>>>>> and in the test harness, both of which are as far as I'm aware relatively >>>>>> unmaintained. Equally, we could treat unversioned URLs as the latest known >>>>>> version, but that again involves changes in both. (There appears to be >>>>>> separate code to group tests into spec sections in both the build system >>>>>> for the index pages and in the test harness for its results display, rather >>>>>> than the latter relying on the build system for the grouping.) >>>>>> >>>>> >>>>> Where is http://test.csswg.org/harness/ maintained? It ought to be a >>>>> 1- or 2-liner fix that we could do at the same time as removing version >>>>> numbers from directories. >>>>> >>>> >>>> It's all in http://hg.csswg.org/dev/harness/. It's also far from a >>>> 1-or-2-liner fix. >>>> >>>> http://hg.csswg.org/dev/harness/file/3e982ede4ca5/ >>>> pages/ResultsPage.php#l433 is the code that generates the results >>>> table, and the codepaths using writeSectionRows are the harder ones to fix >>>> (and the default codepath). >>>> >>>> You also need to change https://github.com/w3c/ >>>> csswg-w3ctestlib/blob/master/Indexer.py to put unknown anchors >>>> somewhere in the index in the first place and then change >>>> http://hg.csswg.org/dev/harness/file/3e982ede4ca5/ >>>> python/SynchronizeSpecLinks.py so that these reach the database the >>>> test harness works from. >>>> >>>> And I've probably missed something, because that's all I found looking >>>> last night without looking overly hard. >>>> >>> >>> Alan, Florian, other CSS WG folks, is there anyone that understands this >>> code still? It seems to me that even if we try very hard to not break >>> what's currently working, then over time it would break as sections change >>> names, or as typos occur. Without a tool that can automatically check the >>> whole test suite for being in the correct shape, it will be hard to keep it >>> from bitrotting. >>> >>> On the other hand, if such a tool did exist, then I think there's a >>> pretty easy fix here: make a best effort to keep things in good shape with >>> the line, but run the tool itself when it's needed to take a spec to REC >>> and fix the problems it reports then. >>> >> >> Having spoken to him on IRC, as I understand it Peter (Linss) is willing >> to amend it and the build system to create an "unknown" section in both the >> build output and the harness (and he's the one who's touched both most >> since the CSS WG adopted the system from whichever group had originally >> created it). >> >> Adding a "dynamic lint" tool to find tests with unknown sections is >> something we can quite easily do, but for obvious reasons we can't block >> any PR on it (even limiting it to a per-file basis, you still want to be >> able to change obvious badness in a test without updating the link). >> >> So what I believe we have rough agreement on: >> >> - Unversioned directories, probably under css/ to make the lint rules >> simpler and easier to understand for contributors. >> >> >> - Keep requiring <link rel=help> for CSS WG stuff. >> >> >> - Require versioned spec links. >> - Don't require links to specific spec sections, but encourage them >> (but don't block on it indefinitely!). >> >> I think the only actionable items for us here are to move everything, to >> change the lint to require versioned spec links, and to change the build >> system to include unknown anchors (I think this is literally just deleting >> an if block); and then the harness changes actionable by Peter? >> > > Awesome, we should do it! > > About the location, I guess we're sticking with css/ so that we don't have > to tackle the question of css/vendor-imports/ right now? > Mostly because I feel strongly that it's easier to tell contributors "if you're dealing with the css directory, there's these extra lints and extra rules" v. "if you're dealing with a directory that maps to a CSS spec, there's these extra lints and extra rules" (esp. now some former FXTF stuff is now in the CSS WG). /g
Received on Monday, 25 September 2017 11:14:33 UTC