Re: checklink: Disable Robots

Hi,

I was wanting to disable robots because we have a huge website and are 
wanting to run a link checker throughout the entire site to verify that 
all of the links work. However, we want to ensure that no other robots 
are allowed to access some of our files, so we still keep all of our 
files and folders in robots.txt.

-Tim Burkhart