- From: CSS Meeting Bot via GitHub <sysbot+gh@w3.org>
- Date: Wed, 30 Aug 2017 16:40:17 +0000
- To: public-css-archive@w3.org
The Working Group just discussed `Test Metadata`. <details><summary>The full IRC log of that discussion</summary> <dael> Topic: Test Metadata<br> <dael> github: https://github.com/w3c/csswg-drafts/issues/1730<br> <dael> astearns: You were putting tests in that had the rel metadata?<br> <dael> Florian: I was moving tests from UI3 to UI4 and I fixed the rel and fixed the file path b/c my memory of when we agreed to merge repos rel was agreed to be optional because same infor was in path<br> <dael> Florian: I niavely updated the path and was told not to do that. Now I understand they prefer when rel is there and hte path doesn't change.<br> <dael> Florian: They want rel=help and the path to be accurate, but higher preference is path to not change. The preferred way to do this is unversioned spec in the path. I thought hte path must encode location and it's only a should<br> <dael> Florian: Currently our tooling relies on rel=help, but I thougth we were eventually supposed to update tooling.<br> <dael> astearns: Also when we imported CSS test to web platoform we didn't follow the spec name folder, it's all in CSS.<br> <dael> Florian: And leveled spec names.<br> <dael> astearns: Right. So I think the point where w fully use path as encoding for the test is when we start adding tests for new spec outside of css directory<br> <dael> Florian: I think there is already a few.<br> <dael> Florian: Key point for me is to understand if it's just me who misunderstood or if it's the whole group. We can't programatically rely on the path or rel-help being there.<br> <dael> Florian: If it' sjust me, fine.<br> <dael> plinss: Clarification. Our tooling depends on rel=help. wpt was planned to have path as encoding and beyond the spec having sub folders. Some structure. THere was a plan to agment our tool to fallback to path, but that's on hold until path name is reliable.<br> <zcorpan_> Currently top-level css directories: css-backgrounds css-cascade css-font-display css-font-loading css-fonts css-paint-api css-scoping css-timing css-typed-om cssom-view cssom<br> <dael> flackr: And there's no plan for that. Spec names unversioned are sorta expected to be reliable, but unversioned not and section level will be more of the same. When a piece of spec moves there's resistance to changing the path.<br> <astearns> s/flackr/florian/<br> <dael> astearns: And other outside tooling relying on that means we can never rely on path. We need a requirement to have the rel=help links.<br> <dael> Florian: But that goes against moving outside css directory<br> <dael> plinss: No. The css directory was there because that was easy for the transition. Our tooling doesn't care about the path at all if there's a rel=help link<br> <dael> Florian: But a wpt will not make rel=help manditory<br> <dael> gsnedders: It's manditory in css subdirectory<br> <dael> Florian: Right.<br> <dael> Florian: But we're not mandating the subdirectory<br> <gsnedders> s/manditory/mandatory/<br> <dael> fantasai: Do we want to transition outside css directory?<br> <dael> plinss: There are already some. There are some people moved out, there were some outside already. I don't really care where they are.<br> <dael> Florian: I kidna came into that...on the one end I think we strongly require this and they strongly require this to be option and we apperently had not agreed.<br> <dael> astearns: I don't think we decided rel=help is mandatory to check in tests. We don't want a fail if it's not there, but we strongly prefer it. As tests get run we're going to add rel=help meta data to those<br> <dael> fantasai: We decided to make rel=help optional because it would be replaced by file path. If hta'ts not the case having it optional makes no sense. We need to know what's being tested.<br> <gregwhitworth> Agree with Elika<br> <dael> Florian: What I was told is that moving thigns alnog the rec track was a known goal for the wpt. Though it's better to have meta data it still works as a test and it's better to have tihs than no tests.<br> <dael> fantasai: We agreed to have our tests in their repo, but we shouldn't drop our goals.<br> <dbaron> I agree with the position -- having regression tests is useful even if they're not used for advancing on the REC track.<br> <dael> astearns: While our tooling isn't going to use anything in the path, we are still able to rely on top level css folder or top levels that corrispond to our tests to see it is css and if it's lacking rel it can be fixed.<br> <fantasai> dbaron, sure, but I expect it's going to be used as an excuse for people to be sloppy and putting more work on us<br> <fantasai> dbaron, it's way easier for the test writer to associate at test with a spec than for us to go back and do so retroactively<br> <dael> astearns: I'd prefer we keep it optional in that ntohing will stop someone from checking is a test but still strongly preferring it so that we have the practise of checking in the rel=help<br> <fantasai> dbaron, also people should COMMENT THEIR CODE, and that includes tests, and the WPT folks insisting that tests be uncommented regression dumps while also believing that function declarations should have some kind of commenting is hypocritical<br> <dael> gsnedders: If I can summerize, two parts. 1 there was some plan to add a per directory file to preserve the mapping to specs even if things got moved around. The other thing was I thought there was some agreement to move away to the tooling people were using for impl tests and wpts<br> <fantasai> dbaron, tests are maintained code, too<br> <dael> plinss: We have to rely on the tooling we have until there's a tool that meets our needs.<br> <dael> Florian: And I don't think there's a preception that we require our meta data b/c w3c, but becuase it makes our lives easier. I've been told we're just imaging requirements from w3c when they're not.<br> <dael> plinss: We've never done our testing infestructure b/c a w3c mandate. They only mandate we have tests. How we do that is up to us.<br> <gregwhitworth> s/imaging/imagining<br> <dael> astearns: fwiw there was smoeone complaining on whatwg irc about an undocumented test.<br> <dael> fantasai: This is a general problem. There are people that think web platform is a regression dump, but there are other people that have to go back and maintain that code. We're those people We have to look at the code.<br> <dbaron> fantasai, There are also regression tests where it's not clear what the specs say, but we actually do have interop and should keep it.<br> <dael> astearns: I don't have a good idea of how many tests we have in wpt that are css tests and lack rel metadata. Does anyone?<br> <dael> gsnedders: In principle everything in css subdirectory does. IT's a q of howmany are outside.<br> <dael> astearns: And how many have meta data<br> <dael> gsnedders: Few outside the subdirectory have it, I think. This is my impression, I don't have numbers.<br> <dael> astearns: Does the current tooling require rel to check in to css ubdirectory?<br> <dael> gsnedders: It does<br> <dael> astearns: So it won't spread in css subdirectory. And maybe we can turn it on elsewhere.<br> <fantasai> dbaron, sure, and those should be filed as bugs against the specs, preferably with a link to the test in question so that we can update it as necessary<br> <dael> Florian: But ther'es a plan to move out of subdirectoyr.<br> <dael> astearns: As we spread out we can spread the requirements to the directorys wil our tests.<br> <dael> Florian: gsnedders do you think it's welcome?<br> <dael> gsnedders: I guess not.<br> <dael> tantek: What's the difference between sub level and top level that's not ours?<br> <dael> gsnedders: The status quo is we have this one top level with a bunch of different requirements to everything else. I think it's the only one that has any different actual check in level req.<br> <gsnedders> s/tantek/gregwhitworth /<br> <dael> Florian: Based on the discussions I've had the rest outside csswg is for the web community and if they want to write tests they shouldn't do our sub directory.<br> <Florian> s/shouldn't do our sub directory./shouldn't have to follow our unusual rules/<br> <dael> astearns: I would guess the policy for a top level should be dictated byt he owners. THe cssom tests are outside the directory. so Simon are the tests there using rel links? And would you welcome having that req for those folders?<br> <dael> zcorpan_: They are currently outside CSS. I believe they mostly have meta data<br> <fantasai> How are people supposed to cooperate on test coverage if everyone's writing tests in their own little corner and not looking at what else is in the repo? Part of the point here is not to duplicate work by having each implementer write its own version of the same exact test.<br> <dael> zcorpan_: Exception is prob there are test using like .window.js<br> <dael> zcorpan_: I ususally at least make sure the path [missed]<br> <dael> astearns: That's part of your review to make sure the rel links are there?<br> <bkardell_> present + bkardell_ (but late)<br> <zcorpan_> I don't know how many of the tests have metadata, but the tests were in css/ before so presumably most have metadata<br> <dael> astearns: Okay.<br> <zcorpan_> Yes<br> <dael> gsnedders: I think some of the cssom tests were already in wpt before the merge<br> <dael> gsnedders: Therefor those might not have the meta data. That's why they live outside css.<br> <dbaron> OK, I guess I'm fine with requiring rel=help<br> <zcorpan_> When I write tests I try to make sure either there's a rel-help or that the directory structure matches which spec is tested<br> <dael> fantasai: I think it's fine if cssom since they're different. IF we have a sep top level for every spec we'd have some test outside css subdirectory and some within so it makes it hard to find things. Also that's a lot of subdirectories so people finding what they're looking for is hard.<br> <Florian> q+<br> <zcorpan_> https://github.com/w3c/web-platform-tests/blob/master/html/README.md is the policy for html/<br> <dael> astearns: I agree it's messy for some ina nd some out, but it's the current reality. We could not add more to it but adding new tests in css. IT sounds to me like we want to continue to require rel meta data links on all tests inside css directory going forward. Doesn't seem like we can change that and it's helpful to have<br> <dael> astearns: We can look into requiring the rel link in other folders for tests outside css folder<br> <astearns> ack Florian<br> <fantasai> There are less than 200 non-CSSOM tests outside the css/<br> <dael> Florian: Questions. 1) if it annoys people when files move we should stop that so when we get close to rec and an at risk feature moves moving them is not liked. unversioned folders solves that. Should we start doing that? If yes, when?<br> <zcorpan_> (I think for Chromium there's no problem with moving tests, the tooling should figure that out. Might be different for other vendors.)<br> <gsnedders> fantasai: including almost all of Houdini, IIRC<br> <Florian> q+<br> <gsnedders> zcorpan_: (expectation data hardcodes the path)<br> <dael> astearns: Any thoughts on moving to non-versioned folders?<br> <dael> fasi"m okay with that.<br> <dael> s/fasi"m/ fantasai: I'm<br> <dael> Florian: If we do it in one shot and warn people it might be okay<br> <zcorpan_> gsnedders: (I was informed that the downstream updater figures out expectations for moved tests.)<br> <dael> gsnedders: There's some consensus it's okay to do these thigns in big moves to get rid of cases where css uses policies not used byt he rest fof the repo<br> <dael> Florian: THat's pref? moving css/cssfoo3 or move everything to css/foo all at once?<br> <dael> gsnedders: There's a part of me in favor of keeping status quo until we have different tooling and them move to top level at once.<br> <dael> fantasai: If we want to move to havin the path encode data we should do that at same time. But if we drop versions we can't do subscriptiosn reliably because they'll change level to level. Some will be same, some won't.<br> <dael> fantasai: I think it'll be tricky to have unvision module directorys but also do waht to have per section sub directories. We can do one or the other but not both.<br> <dael> Florian: I think we might want subdirectories per topic, but if we can't move the shouldn't be fine grained.<br> <dael> fantasai: Agree. We should just have per module directories on loevel. It would be a good idea to move the tests outside css subdirectory in and then maintin that for all tests. If we put them top level it'll be 60 top level and be hard for people to find all css tests<br> <dael> fantasai: I think it makes the repo more usable if we don't have all 60+ modules scattered across the top level<br> <dael> astearns: gsnedders can I give you an action to look at moving to unversioned folders in one go to avoid file movemenets in the futuer? Within the css subdirectory?<br> <dael> gsnedders: I guess<br> <dael> Florian: Until we have the answer, should we start making unversioned subdirectory to avoid making it worse for new tests?<br> <dael> astearns: I think it makes sense for new tests<br> <dael> fantasai: If we're going to move a bunch people will have to deal. IF we don't move we'll have to move those things back.<br> <dael> astearns: Perhaps new test suite if you have to make a new folder, make it unversioned.<br> <astearns> ack Florian<br> <dael> Florian: Question 2) If we move out of css subdirectory for purpose of no logner following css rules, what do we lost beyond rel=help?<br> <dael> gsnedders: I can't remember all. There's the check for unique files names within test suite. We have rel=help. There's another I can't remember<br> <dael> Florian: These are things we only care about b/c build tool relies on them. Correct?<br> <dael> gsnedders: Yes. All CSS only events are there to keep the build system working.<br> <zcorpan_> css/ also can't use .window.js, .worker.js, .any.js, since they can't encode rel=help<br> <zcorpan_> though adding ability to encode a spec link in those seems possible<br> <dael> plinss: Beyond that the build system org tests into per spec suites and generates other versions of tests. OTher tooling relies on that outpul like test harness and spec annotations. Also if we ever want anything to say what tests a spec change may break. So I keep hearing we might move, but if we lose the meta data we lose these extra capabilities.<br> <gsnedders> zcorpan_: also that doesn't work in the build system, because they don't use wptserve<br> <dael> Florian: This idea is that other people get by without this and why can't we. I don't know why we should change.<br> <dael> plinss: We care about taking specs to rec. It's a non goal of wpt so do not expect support for that goal.<br> <dael> astearns: Other people also require a test when you change aspec.<br> </details> -- GitHub Notification of comment by css-meeting-bot Please view or discuss this issue at https://github.com/w3c/csswg-drafts/issues/1730#issuecomment-326048612 using your GitHub account
Received on Wednesday, 30 August 2017 16:40:14 UTC