- From: Ville Skyttä <ville.skytta@iki.fi>
- Date: Tue, 20 Apr 2004 09:28:46 +0300
- To: QA Dev <public-qa-dev@w3.org>
On Tue, 2004-04-20 at 01:06, olivier Thereaux wrote: > I have just installed LWP 5.79 on qa-dev. Maybe we can check for a day > or two whether that causes no problem with either the validator or > checklink, and proceed with a beta (with 5.79 installed on w.v.o) > within a couple days? Sounds good. > I still find it frustrating that the link checker HAS to follow > robots.txt and telling users to check links manually, even in non > recursive mode, but I reckon I am against the majority here, and I > should probably just accept I am wrong and shut up ;) FWIW, I am not a huge fan of it either. A doc snippet (and a pointer to it from the "forbidden by robots.txt" explanation) how to allow the link checker in /robots.txt would probably be a good idea, I'll look into it.
Received on Tuesday, 20 April 2004 02:28:52 UTC