Re: checklink:

2013-08-22 21:50, Andresen, Nancy - DOT wrote:

> I used to use your service all the time, but today I get a “forbidden by
> Robots” error for every page I try.  For example:
> _
> Can you tell me what has changed?

The exact message is "Error: 403 Forbidden by robots.txt". The reason is 
that the robots.txt resource on your server,
disallows all robots, and the Link Checker regards itself as a robot and 
honors the Robots Exclusion Standard.


Note that the robots.txt file also affects Google. If you e.g. Google 
with the page URL
the top search result will have no description to show; instead it has 
the explanation "A description for this result is not available because 
of this site's robots.txt".

So consider contacting your site admin asking them to reconsider the 
usefulness of excluding all robots.


Received on Sunday, 25 August 2013 18:27:59 UTC