W3C home > Mailing lists > Public > www-validator@w3.org > June 2010

Re: Robots.txt with Link checker

From: David Dorward <david@dorward.me.uk>
Date: Tue, 8 Jun 2010 13:18:17 +0100
Cc: <www-validator@w3.org>
Message-Id: <0DBF636A-BB19-406E-9D5A-8DF03DBAC417@dorward.me.uk>
To: webmaster@groenhuis.nu

On 6 Jun 2010, at 22:50, webmaster@groenhuis.nu wrote:
> As webmaster for the website http://www.groenhuis.nu i want our website to
> validate. The link checker tool is "complaining" it can't acces some
> javascript files. Is this something i should correct. I put these files
> into the robots.txt because i think it is not nessesary for searchengines
> to index these files.


Well, yes. You told robots not to go to your JS. The link checker is a robot and it is doing what you told it to do.

-- 
David Dorward
http://dorward.me.uk
Received on Tuesday, 8 June 2010 12:18:57 GMT

This archive was generated by hypermail 2.2.0+W3C-0.50 : Wednesday, 25 April 2012 12:14:41 GMT