W3C home > Mailing lists > Public > www-validator@w3.org > April 2004

Re: checklink: htmlhelp is forbidden

From: Nick Kew <nick@webthing.com>
Date: Sun, 4 Apr 2004 22:33:01 +0100 (BST)
To: www-validator@w3.org
Message-ID: <Pine.LNX.4.53.0404042227000.969@hugin.webthing.com>

> > I don't know why htmlhelp.com blocks the link checker, but I wouldn't be
> > surprised if it was something to do with the way it (the link checker)
> > ignores the robots exclusion standard.
>
> Yes, that's why.  I banned W3C-checklink after it overloaded our server,
> preventing legitimate users from accessing it.  W3C-checklink had been
> spidering resources that were disallowed in robots.txt.

A while ago I had email from Liam drawing my attention to the fact
that I'd introduced a similar bug into htmlhelp's own link checker.
I've since fixed that, and round tuits permitting, I'll take a look
at the same issue in checklink.

It can be very easy to overlook bugs when you don't have someone
complaining about them at a convenient time.

-- 
Nick Kew

Nick's manifesto: http://www.htmlhelp.com/~nick/
Received on Sunday, 4 April 2004 17:33:33 GMT

This archive was generated by hypermail 2.2.0+W3C-0.50 : Wednesday, 25 April 2012 12:14:13 GMT