Re: Error: 403 Forbidden


The linkchecker sends the following User-agent field:
> User-Agent: W3C-checklink/4.3 [4.42] libwww-perl/5.805
and it seems that your server is configured to deny access to any 
user-agent that identify itself as "libwww-perl". Hence, linkchecker is 
denied access to your website.

As you asked for it, the IP from which comes linkchecker request is: At least, that’s the IP it used last time it checked my 
site (search your logs for W3C-Checklink).

Additionally, I suggest you review your robots.txt file:
  - It is sent as 'text/html', but the normal media-type for this file is
  - The last three records don’t follow any "User-Agent" record, hence if
    I understand the standard correctly, they don’t apply to any robots.



Received on Tuesday, 23 September 2008 18:34:02 UTC