Re: Unwanted robot accesses from your site

Hi Bjoern.

On Wednesday, Dec 25, 2002, at 13:48 Asia/Tokyo, Bjoern Hoehrmann wrote:
> Why doesn't checklink qualify as a robot?

My own definition of a robot is that it retrieves some data (the 
documents) or metadata (indexing). I may be wrong.

In any case I don't think checklink, even in recursive mode, should 
follow the robots directives (noindex is irrelevant, and nofollow would 
make it useless...). I'm interested to hear opposite arguments, though.

-- 
Olivier