Re: checklink: htmlhelp is forbidden

> > I don't know why htmlhelp.com blocks the link checker, but I wouldn't be
> > surprised if it was something to do with the way it (the link checker)
> > ignores the robots exclusion standard.
>
> Yes, that's why.  I banned W3C-checklink after it overloaded our server,
> preventing legitimate users from accessing it.  W3C-checklink had been
> spidering resources that were disallowed in robots.txt.

A while ago I had email from Liam drawing my attention to the fact
that I'd introduced a similar bug into htmlhelp's own link checker.
I've since fixed that, and round tuits permitting, I'll take a look
at the same issue in checklink.

It can be very easy to overlook bugs when you don't have someone
complaining about them at a convenient time.

-- 
Nick Kew

Nick's manifesto: http://www.htmlhelp.com/~nick/