[Prev][Next][Index][Thread]

Re: Benefits of :<META NAME="ROBOTS" CONTENT="NOINDEX,FOLLOW">



> <META NAME="ROBOTS" CONTENT="NOINDEX,FOLLOW">

FOLLOW refers to extracting links from a page, INDEX refers to
extracting content and indexing it. For example, if you have

 page1.html:

   <a href="page2.html">here is another page</a> for you

When a robot sees "NOINDEX,FOLLOW", it ignores the text "here is another
page for you", but does go and retrieve page2.html.


> The info. I've read doesn't describe the benefits of the "noindex, follow".

See http://info.webcrawler.com/mak/projects/robots/exclusion.html#meta

-- Martijn Koster, m.koster@pobox.com


References: