W3C home > Mailing lists > Public > www-validator@w3.org > July 2007

[checklink] How to override robots.txt?

From: Stephan Windmüller <windy@white-hawk.de>
Date: Tue, 17 Jul 2007 12:39:49 +0200
To: www-validator@w3.org
Message-ID: <20070717103949.GA7797@speutel.de>
Hello!

We want to use checklink to search for broken links on our site. Of
course there should be only a warning mail when something went wrong.
Unfortunately this is also true when external links are disallowed by a
robots.txt.

Is it possible to override or disable the robots.txt-feature? If not,
how can I prevent checklink to list these adresses as errors when using
"-q"?

TIA
 Stephan

Received on Tuesday, 17 July 2007 18:14:35 GMT

This archive was generated by hypermail 2.2.0+W3C-0.50 : Wednesday, 25 April 2012 12:14:25 GMT