[00:55] --- bjoern_ has changed the topic to: W3C QA Tools || Markup Validator v0.6.7 Released || current meeting: now || [00:58] let's skip the AI and other such stuff for today, esw being down [00:58] xover, bjoern_: given that S.P.O installs cleanly, if I were to try rewriting validator-lite in Perl, what would my chances be? [00:58] validator-lite is OpenSP+GUI? [00:59] bjoern_: more-or-less [00:59] and if I can do it in Perl it becomes more platform-neutral [00:59] You could also use one of these GUI kits... [00:59] it's currently GTK [00:59] Problem is that people might not have Perl installed [01:00] other than that you should not have problems doing that [01:00] which is fine on *X (including MacOSX) [01:00] Put another way; S:P:O ain't gonna be your problem, I don't think. [01:01] ok, so if someone will pass me a round tuit .... [01:01] speaking of status of SPO, what's the general opinion on installing sp from CVS and use SPO there? [01:01] * yod miserably failed building sp today. twice [01:01] but I don't doubt someone else could do it [01:01] yod: version? [01:02] ...on qa-dev? [01:02] CVS€ :-) [01:02] niq CVS with -r 1_5_branch or something like that [01:02] hmmm [01:02] what was the problem? what gcc? [01:02] (details in a mail by bjoern) [01:03] bjoern_, 3.3.4 on my deb box [01:03] well, I had no problem compiling it after doing all the autoconf stuff [01:04] on debian testing [01:04] with gcc 3.3.4 [01:04] ran just fine, the 70 minutes it ran... [01:05] I've just been following the BUILDING instructions [01:05] hmm must be a dependency problem, I had a warning on ALL_LANGS (I think) during autoconf [01:05] should probably not have ignored it [01:06] I can try again, but my lack of luck should not preclude us from building it on qa-dev - we all have sudo AFAIK [01:06] I think we should run OpenSP-cvs on qa-dev [01:06] is "opensp_1_5_branch" the correct one in CVS? No recent ChangeLog or NEWS entries there... [01:06] yes [01:07] * bjoern_ did not update the changelog [01:07] xover failed to tell me how this is supposed to work... [01:07] :-) [01:07] «Well, you start by fedex'ing oodles of cash to this address...» [01:08] * bjoern_ glad to be no OpenSP project admin... [01:08] other opinions re OpenSP-cvs on qa-dev? [01:09] volunteers to install it? [01:09] * xover can put it on his' TODO list, but... [01:10] If someone wants to point to a source tree, I'll look at it. [01:10] but I don't want to faff about with CVS [01:10] niq + SF.net CVS == Trouble! :-) [01:11] you mean you want a .tgz? [01:11] a .tgz would do [01:11] I'll give you one [01:11] the plan is to not "overwrite" the system one, but install to /usr/local or something, right? [01:12] dunno... [01:12] scop: that's how I see it [01:12] well, if I build it'll be under /home/nick [01:12] ./configure --prefix=/usr/local sounds like a good idea, yes. [01:12] then if that's successful it can be exported [01:12] xover: with opensp you can avoid ./configure [01:13] do "../opensp/configure" from a separate directory instead. Cleaner [01:13] especially when things have to be hacked [01:14] * scop plans to roll fedora core packages of opensp CVS soonish, btw [01:14] I will send niq a tgz [01:14] don't send ... [01:14] send a filepath to a .tgz on qa-dev ... [01:14] * bjoern_ stops `cvs co`... [01:15] ah, sorry bjoern_ [01:16] while we're here, other things that need to be installed on qa-dev? [01:16] I suppose SPO will be next [01:16] Test::Exception and Class::Accessor would be good [01:16] and XML::LibXML for our test suite [01:16] and other things [01:17] * xover notes everyone should have access to do this directly... [01:17] indeed [01:18] * bjoern_ refuses sudo rights! [01:18] niq, there is a osp source tree in /home/bjoern/osp-cvs/sp [01:18] just copy that to your home [01:19] apt-get install libtest-exception-perl libclass-accessor-perl libxml-libxml-perl libxml-sax-perl [01:19] bjoern_: done [01:19] 'k, I'll remove it then [01:20] * yod doing the apt-get'ing in the bg FYI [01:20] not sure why bjoern refuses sudo right, no big deal [01:21] well, without root rights you cannot mess up root stuff :) [01:21] * niq doesn't think he can sudo (can't remember password) - maybe bjoern_ likewise [01:21] oh [01:22] it doesn't ask for password :-| [01:23] okay, any other qa-dev install stuff? [01:24] moving on... [01:24] what's the next step plan for m12n? [01:25] I've posted some stuff to qa-dev [01:26] one of the next steps would be the 0.7.0 release [01:26] we also need to fine someone who re-implements eg outline() on top of SAX [01:27] bjoern_: that's perl-sax? [01:27] yes [01:27] see perl-xml.sf.net [01:28] * niq thinks [01:28] I think we generally lack a plan ATM [01:28] indeedie [01:29] so shall we hatch up a dastardly plan while xover is mostly-elsewhere? [01:31] I should probably know this, but... when do we start to use S:P:O or SAX in the first place? in or after 0.7.0? [01:31] * xover hears his IRC client beep, notices niq plotting dastardly plans, and goes back to being content that all is well with the world... [01:31] well, that depends on our 0.7.0 release plan [01:31] and we don't have a precise one yet [01:31] What odds do you give me Bjoern would hunt me down and _kill_ me if I tried stuffing S:P:O into 0.7.0? :-) [01:32] one problem here is that I see a release as a means to make improvements available to our users and xover more as a great opportunity to fix all the bugs... [01:32] ok, so we're missing two plans: m12n and 0.7.0? [01:32] yeah... [01:33] There is... debate... about the specifics of 0.7.0. [01:33] yod posted on the subject back in July ... time to revive that? [01:33] ie m12n [01:33] http://www.w3.org/Bugs/Public/show_bug.cgi?id=856 [01:34] 21 bugs / RFEs that block 0.7.0 [01:34] http://www.w3.org/Bugs/Public/show_bug.cgi?id=856 [01:34] gah, slow [01:34] we should hash out which of them *really* block 0.7.0 [01:34] and see how to get them addressed [01:34] afaict from earlier discussion, xover was ok with trimming the list [01:34] most of them are bugs that were marked "review around 0.7.0" [01:35] * xover _has_ been trimming the list... [01:35] xover, does that mean all bugs that remain blockers are things you feel very strongly about addressing them in 0.7.0? [01:35] There's at least another two that I'll probably nuke fairly quickly (63 and 75). [01:35] and a lot of them have been fixed in the provess [01:35] cess [01:36] yes, we started with > 40 blockers... [01:36] There's 2+ that will probably get nuked if nobody deals with them as I don't think I'll have the time (docs, CSS). [01:37] Another handfull will get FIXED by the same or related checkins (that I'm currently working on) spawning form the new config system. [01:37] ETA? [01:37] Hard to predict right now; work is being obstinate ATM. [01:38] There's a few genuine bones of contention in there — e.g. Tabtastic — that will probably end up being nuked just because they're contentious. [01:38] tabtastic++ [01:38] tabtastic-- [01:38] bjoern_-- [01:38] Exactly! [01:39] But in general... We may have started out with 40+ blockers, but I don't think we've bitten over more than we can chew. [01:40] xover, what do we do with the blockers yod and/or I think are better addressed later? [01:40] many of them are still blockers [01:40] (http://www.w3.org/Bugs/Public/show_bug.cgi?id=856 still) [01:41] Well, four of the bugs on your list of "not for 0.7.0" are CLOSED FIXED already... [01:42] (well, and a WONTFIX) :-) [01:42] considering that there are about 20 in my list that's not much... [01:44] I've been focussing on landing one key checkin; after that I'd planned to go over the list of blockers again and have a round on the mailinglist about them. [01:46] so our plan for 0.7.0 is "we'll see"? [01:46] Mostly because several of them are interrelated and we can close a bunch of them in one go, or more cleary tell that the rest don't fit for 0.7.0. [01:48] by a bunch of them you mean 2 or 3? [01:48] most of them do not seem related to me... [01:49] What do other people think, what is more important, that we attempt to fix all these bugs now or that we start cleaning up the code? [01:50] Why do you have an "or" in there? [01:50] * yod gave opinion already, think some of the bugs can be fixed, would like to see some more discussion on #856 [01:51] * xover counts 5 bugs that will be resolved by one change (directly or as a result of), and another couple that are similarly related. [01:51] it does not have to be an either/or, the question is to agree on how much [01:51] how about agreeing on a timeframe and postpone bugs not fixed within that scheduled time? [01:51] All these bugs are steps on the path to cleaning up the code. [01:52] SOme more than others (*chough*Tabtastic*chough*), but... [01:52] I think I disagree that stuffing more code into check or messing with code in check really helps cleaning up the code. [01:53] 0.6.7 is now almost 2 months old and not much happend since [01:53] I would estimate that addressing all the current blockers would take several months [01:54] and I've also pointed out that addressing a number of them will be easier [01:54] once we use Encode, S::P::O, etc. [01:55] scop, did you have a look at 856? [01:56] bjoern_, yeah, skimming it and scratching my head, trying to form an opinion... [01:56] * xover notes one of the blockers there is assigned to Ville... [01:57] well, at least I can take care of bringing the installation docs up to date [01:57] xover, which one? [01:57] #70 [01:59] maybe we should go through the bugs which yod/me don't think that they should block 0.7.0? [01:59] Installation docs (#78), and print stylesheet (#109), will likely get nuked if e.g. yod doesn't do it. [02:00] most of them will likely get nuked if no one takes care of them [02:00] but that's not really a good plan [02:01] I should also note that a number of bugs that *really* block 0.7.0 are not blockers yet [02:01] like that our verbose messages are not included in the output [02:01] Well, then why haven't you put them in the list yet? [02:01] xover: re- 109, I'll happily do it, but you seem to have assigned it to you [02:01] that's also a problem btw [02:02] don't assign a bug to yourself unless [02:02] you plan to fix it in the next two weeks or so [02:02] could be bugzilla's default , dunno [02:02] It is. I'm the default component owner for everything. [02:02] default is NEW or UNCONFIRMED [02:03] default owner, that is [02:03] www-validator-cvs is the QA-contact. [02:03] anyway, I'll take #109 [02:03] http://qa-dev.w3.org/wmvs/HEAD/feedback.html has bugzilla links btw [02:04] * yod would like to close this topic [02:04] I suggest: [02:05] niq and scop to chime in on that thread, give what they think are their priorities [02:05] hmmmmm [02:05] we drop anything that is agreed to be simpler with m12n [02:05] simpler/better done with [02:05] hmmmm .....# [02:05] hmmmm? [02:06] we want a list of what we think MUST be done before we can release 0.7.0 [02:06] ack [02:06] * niq recollects you posting on m12n a while back, and me trying to take it up ... [02:06] then I lost momentum [02:06] niq: there are several aspects of m12n that we can discuss [02:07] indeed [02:07] * xover notes Bjoern uses MUST where he would use SHOULD... [02:07] but we can make progress on 0.7.0 as we work these out [02:07] http://lists.w3.org/Archives/Public/public-qa-dev/2004Jul/0010.html [02:07] BTW, yod, Bugzilla keeps wanting me to log in. Proxying fucking things up? [02:08] * bjoern_ had that too... [02:08] but it worked out for single changes etc. [02:08] it does that to me too, indeed [02:09] and re MUST, must in the sense that it would otherwise significantly reduce the quality of the product [02:09] if we have completed all these bugs and there are more things that could go in, no problem [02:09] but we need a plan [02:09] especially so that people can take bugs [02:09] I am overwhelmed by the number of bugs [02:10] and in fact I am keeping a list of all the things that really require attention [02:10] * xover repeats: Why aren't these in Bugzilla and added as blockers? [02:10] so that I don't waste most of my devel time to figure this out again and again [02:10] I am talking about things inside bugzilla [02:11] so do I understand correctly that [02:12] niq and scop will add their opinions to bugzilla, [02:12] and then yod will go through things that overlap and remove the blockers? [02:14] fine with me [02:14] niq? xover? [02:14] ok [02:15] * xover is waiting to hear from yod... [02:15] yod is waiting to hear from xover... [02:16] xover: hear form me on? the fact that we should remove blockers most of us think should be done later? [02:17] then yes, I agree [02:17] Bjoern's question: "so do I understand correctly that" + the next two lines. [02:18] Ok. Then I'll consider myself outvoted on this issue. [02:19] there's no "vote", at least not yet, not until niq and scop give their opinion on the #856 list [02:19] the proposal is to review the list together and decide based on that [02:20] I don't see why that's a problem [02:20] well, we've resolved to focus on the things we agree to focus on and then see, there's really no outvoting in that. [02:20] my tuit is misshapen [02:21] speaking of which... is it reasonable to expect hearing from you on this list within a couple of days? [02:21] it does take a bit of time to review the list (although it's gotten thinner) [02:22] for my part, yes, probably tomorrow [02:22] took me about 15 minutes FWIW [02:22] days: unlikely. next week much more likely [02:23] bug me @weekend [02:23] I'll do that. Checklink [02:25] two things on this topic: [02:25] 1- the prototype Ville sent a few days ago (?) [02:26] (http://lists.w3.org/Archives/Public/public-qa-dev/2004Sep/0081.html ?) [02:26] 2- the (minor) complaints about the overall time it takes - and the delay() [02:26] bjoern_, that's it, thanks [02:27] (on the formet, I think the xmlbase filter should be called XmlBase, not XMLBase, but asking about that on perl-xml would be good) [02:27] former [02:27] re CSS::SAC... I am the current maintainer, send patches to me :-) [02:27] on 1-, scop, what kind of feedback do you wish to get [02:27] good question [02:28] * yod started looking around, it's nice, I want to play with it, but guidelines on what to look for would be good too [02:28] I think a good way to proceed would be to /me send a list of encountered things and open issues to qa-dev and/or wiki [02:29] +1, whichever [02:29] bjoern_, re: CSS::SAC: http://rt.cpan.org/NoAuth/Bug.html?id=5187 [02:29] given the state of the wiki host, however, I suggest list for now [02:30] if issues are bug in the current dev version, feel free to enter them in bugzilla too [02:30] if you want people to assign it to themselves, etc [02:30] list is fine. we can of course discuss things here as well now if you want to [02:30] scop, I'll look at that. There are also a number of changes in CVS that haven't made it to CPAN yet... [02:30] scop: rapidly (i.e 20 min max), yes [02:31] that would be good [02:31] ok, off the top of my head: [02:31] what's the "speed" issue you mentioned, yod? [02:31] people dislike not setting servers under heavy load? [02:32] is this specifically for w3.org or more general? [02:32] more general, although the most vocal criticism came from someone within w3c [02:33] any suggestions how to resolve that? [02:33] the basic issue being that in non "verbose" mode, there is no output as the links are processed, and it can take a few minutes if the page has 100+links, which cause the browser to decide it's a timeout [02:33] well, we can reduce sleep to 1/2 seconds or something or have a [x] don't sleep between requests option or [02:34] have something that triggers link checking and sends results per mail or something along these lines [02:34] "don't sleep" in a public instance is not a good idea IMO [02:34] 1/2 second would be an idea, no sleep on configured "private" domain is another [02:34] too much scope for abuse [02:35] emit one whitespace char every now and then in non-verbose mode? [02:35] agreed for public instance, hence my idea of doing it only if configured, and for the private domain [02:36] we could also have an option [_] don't sleep for requests for docs on the same server like the input url [02:36] scop: a bit hackish, but why not. that would not fend off the "it used to be much faster" complaints, but I have hopes that we can convince such people that it's for a greater good [02:37] 1/2 second not doable with vanilla LWP::RobotUA, needs an integer (uses sleep()) [02:37] scop: I checked, no need for an integer [02:37] sleep(60) :-) [02:37] delay() (I think that's the method) takes a folat [02:38] yod, yes, but that's *minutes* [02:38] but sleep only supports > 1sec [02:38] you'd need e.g. select() to sleep in ms [02:38] delay() translates internally into a sleep() call [02:38] ah, hmm [02:38] my bad [02:39] we could also say that you need an explicit useragent:checklink\nallow:/ in the robots.txt [02:39] which would then disable sleeping [02:39] at user option [02:39] or extend the robots.txt format... [02:41] * bjoern_ notes that we are again on the 2 1/2 hours track... [02:41] might make more sense to meet more often but not as long... [02:43] * bjoern_ still awake... doesn't know about the others... [02:43] sort of [02:43] but we started 1/2 late [02:43] hence my taking up this agenda [02:44] awake but undecided; first impressions, explicit_useragent++, extending_robotstxt-- [02:45] (maybe) [02:45] this is a tough problem, conflicting requirements [02:45] we can continue discussion on it by e-mail [02:45] robotstxt - can only use what people know about. not extend [02:46] niq++ [02:46] * niq thinks an online service needs to be limited [02:46] * bjoern_ impressed by the number of people considering that a serious suggestion... [02:47] an online service also needs to take its users needs into account [02:47] public online service is the easy part. [02:47] bjoern_: we all know germans don't make jokes [02:47] [02:48] hmm... right, good point [02:48] I suggest scop/yod send an email to www-validator so we can discuss it there [02:48] if we make it possible to turn the delay off, people who download checklink _will_ do so, and it's back to square one: "checklink kills my site" [02:49] indeedie [02:49] * yod states (again) the question of the delay (or not) is a minor complaint [02:49] this is btw why Liam asked to bump the version number in User-Agent; at that time his servers were under rapid fire from W3C-Checklink/3.06 (or something ancient like that) [02:50] the relevant and accepted issue was about timing out [02:50] well, that's easily addressed by emitting data [02:50] yes, timing out is bad [02:50] precisely [02:51] so the bulk of the issue is easily resolved [02:51] I believe there are some kind of server limits as well, emitting data might not be universally applicable [02:51] whether we want to (re)open the delay can of worms is up to us [02:52] We should have the private site thing and set it to w3.org for the v.w3.org installation [02:52] This allows parallell rapid-fire of w3.org by third parties. [02:53] that would not have to be enabled for the public checklink instance... [02:54] rapid-fire on w3.org not a big issue [02:54] not as big as on non-w3c-sites [02:54] I have logs of discussions with systeam saying it's ok for vwo to pound on wwo [02:54] yod: could become an issue [02:54] niq: how so? [02:55] * niq recently been attacked by a ua trying over 30 validations per second [02:55] which is a lot on my hardware [02:55] yes, the risk is no 0 [02:55] as scop notes, however, the option of a private instance is open [02:56] I hacked up a new module to return robots.txt to them .... [02:56] perhaps a regexp config option "allow delayless operation against these sites" would be acceptable, dunno. I had thought only about a boolean option, which is not good. [02:57] That regex _is_ a boolean. .* [02:58] doh :) [02:58] whatever we do here, it'll always be just changing a few lines to disable that [02:59] and I find it quite odd that someone would use checklink for DoS [02:59] there are many better tools for that [02:59] it does not even to parallel requests, does it? [02:59] Never attribute to malice, that which can be adequately explained by mere stupidity. [02:59] no parallel [02:59] OTOH, faking the user agent to checklink makes sense for an attack [03:00] disabling delay already is a one-line change: $ua->delay($Opts{Sleep_Time}/60) -> $ua->delay(0) [03:01] would "don't delay for hosts in the same domain as this one: yes/no" be useful [03:02] and change local hostname to attacker.victim.com [03:03] Given the lack of good solutions, and the vagueness of the complaints about the status quo, wouldn't WONTFIX be the correct response to the _delay_ issue? [03:03] niq, yes, of course, but that's more difficult than changing delay(0) [03:03] xover: if the "bug" is "delay exists", then yes [03:04] I should note that we will have similar issues for the WMVS once we have a site-validation feature [03:04] we should rather look for some opt-in thing [03:04] site-validation-- [03:04] something that gurantees the server admin allows $x [03:06] That's quite a lot of overhead. Is the benefit worth the investment? [03:06] probably not for checklink [03:07] bjoern_++ [03:07] MHO, delay: WONTFIX or LATER, but timeouts need attention [03:07] any other ideas apart from emitting whitespace? [03:08] you can emit markup like comments etc [03:08] Browser timeouts have been a problem in CGI for a while. There may be established solutions beyond emitting NOOPs. [03:09] or ;P [03:09] that would be similar to what netmechanic.com /toolbox/html-code.htm [03:09] does [03:10] just that they have a progress bar rather than messing with .status [03:11] whoa [03:11] bugzilla uses a multipart response, btw [03:11] (when searching for bugs) [03:12] bugzilla uses agent sniffing for this multipart response [03:12] not sure we want to go there :) [03:12] * scop flees [03:12] Lets do CGI Chat Rooms on v.w3.org! [03:12] anyway... I don't think we'll come up with a solution on this maybe-issue [03:12] we can keep noodling about it, discuss, but not now [03:13] adjourn++ [03:13] hence, yes, I suggest we adjourn for today [03:13] AI(yod): timeout issue -> Bugzilla ? [03:13] yes Adjourned