- From: Martijn Koster <mak@excitecorp.com>
- Date: Wed, 23 Jun 1999 19:22:31 +0100
- To: Don <dledarney@man.net>, www-disw-request@w3.org
> <META NAME="ROBOTS" CONTENT="NOINDEX,FOLLOW"> FOLLOW refers to extracting links from a page, INDEX refers to extracting content and indexing it. For example, if you have page1.html: <a href="page2.html">here is another page</a> for you When a robot sees "NOINDEX,FOLLOW", it ignores the text "here is another page for you", but does go and retrieve page2.html. > The info. I've read doesn't describe the benefits of the "noindex, follow". See http://info.webcrawler.com/mak/projects/robots/exclusion.html#meta -- Martijn Koster, m.koster@pobox.com
Received on Wednesday, 23 June 1999 14:22:15 UTC