W3C home > Mailing lists > Public > www-validator@w3.org > June 2010

Re: Robots.txt with Link checker

From: Ville Skyttä <ville.skytta@iki.fi>
Date: Wed, 9 Jun 2010 21:59:13 +0300
To: www-validator@w3.org
Message-Id: <201006092159.13856.ville.skytta@iki.fi>
On Tuesday 08 June 2010, David Dorward wrote:
> On 6 Jun 2010, at 22:50, webmaster@groenhuis.nu wrote:
> > As webmaster for the website http://www.groenhuis.nu i want our website
> > to validate. The link checker tool is "complaining" it can't acces some
> > javascript files. Is this something i should correct. I put these files
> > into the robots.txt because i think it is not nessesary for
> > searchengines to index these files.
> 
> Well, yes. You told robots not to go to your JS. The link checker is a
> robot and it is doing what you told it to do.

See also http://validator.w3.org/docs/checklink.html#bot
Received on Wednesday, 9 June 2010 18:59:52 GMT

This archive was generated by hypermail 2.2.0+W3C-0.50 : Wednesday, 25 April 2012 12:14:41 GMT