Re: checklink: Disable Robots


Sorry, I have never worked with the robots.txt file before. But I just 
looked at the documentation closer, and I realize that I misinterpreted 
how robots.txt works.

Whenever I took a first quick glance, I just noticed the lines:

Disallow: ...
Allow: ...

And I assumed they were for all robots. Now that I took a closer look, I 
see how to do this. Thanks for the help!


Received on Monday, 23 May 2005 17:08:08 UTC