[Prev][Next][Index][Thread]

Nondeliverable mail



------Transcript of session follows -------
minlee@netsgo.com
The process cannot access the file because
it is being used by another process.

Received: from netsgo.com - 210.115.122.156 by netsgo.com with Microsoft SMTPSVC;
	 Thu, 24 Jun 1999 03:48:35 +0900
Received: from www19.w3.org - 18.29.0.19 by netsgo.com  with Microsoft SMTPSVC(5.5.1775.675.6);
	 Thu, 24 Jun 1999 03:48:28 +0900
Received: (from daemon@localhost)
	by www19.w3.org (8.9.0/8.9.0) id OAA14576;
	Wed, 23 Jun 1999 14:22:22 -0400 (EDT)
Resent-Date: Wed, 23 Jun 1999 14:22:22 -0400 (EDT)
Resent-Message-Id: <199906231822.OAA14576@www19.w3.org>
Message-ID: <19990623192231.A17742@excitecorp.com>
Date: Wed, 23 Jun 1999 19:22:31 +0100
From: Martijn Koster <mak@excitecorp.com>
To: Don <dledarney@man.net>, www-disw-request@w3.org
References: <000801bebdac$aadd3180$e830c8cd@don>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
X-Mailer: Mutt 0.93i
In-Reply-To: <000801bebdac$aadd3180$e830c8cd@don>; from Don on Wed, Jun 23, 1999 at 12:14:49PM -0700
Subject: Re: Benefits of :<META NAME="ROBOTS" CONTENT="NOINDEX,FOLLOW">
X-List-URL: http://www.w3.org/pub/WWW/Archives/Public/www-disw/
Resent-From: www-disw@w3.org
X-Mailing-List: <www-disw@w3.org> archive/latest/80
X-Loop: www-disw@w3.org
Sender: www-disw-request@w3.org
Resent-Sender: www-disw-request@w3.org
Precedence: list
Return-Path: www-disw-request@w3.org

> <META NAME="ROBOTS" CONTENT="NOINDEX,FOLLOW">

FOLLOW refers to extracting links from a page, INDEX refers to
extracting content and indexing it. For example, if you have

 page1.html:

   <a href="page2.html">here is another page</a> for you

When a robot sees "NOINDEX,FOLLOW", it ignores the text "here is another
page for you", but does go and retrieve page2.html.


> The info. I've read doesn't describe the benefits of the "noindex, follow".

See http://info.webcrawler.com/mak/projects/robots/exclusion.html#meta

-- Martijn Koster, m.koster@pobox.com