- From: Conan Callen <ccallen@windowpane.com>
- Date: Wed, 15 Dec 1999 20:00:49 -0500 (EST)
- To: <www-talk@www.cern.ch>
- Cc: <m.koster@webcrawler.com>
Received on Thursday, 16 December 1999 09:03:44 UTC
Hi, I was wondering if there is a way to tell a robot to drop everything it knows about your site and reindex the whole thing? We have recently reworked our whole site and now robots have been requesting pages that are no longer there. It would be real convient if there was a reindex directive that could be place into the robots.txt that would let the robot know that significant changes have occured to this site and it would probably be best to start over from the top (especially if many new pages have been added, and old ones moved or deleted). Im not on this mailing list, if there are any replies please cc my email - ccallen@windowpane.com conan
Received on Thursday, 16 December 1999 09:03:44 UTC