W3C home > Mailing lists > Public > www-validator@w3.org > December 2002

Unwanted robot accesses from your site

From: miim webmaster <xxdpplus@yahoo.com>
Date: Sat, 21 Dec 2002 21:39:51 -0800 (PST)
Message-ID: <20021222053951.44758.qmail@web14910.mail.yahoo.com>
To: www-validator@w3.org

We don't want your robot to visit our site.

We don't use your service, and we don't particularly
like the idea of other people using your robot to
scan our site.

There is a standard for robot exclusion.  Your robot
doesn't follow it.  It should.  All we can do at this
time is use Apache rules to shovel garbage into it
when it visits.  It would save everyone a lot of
trouble if it honored the standard robots.txt.

Do you Yahoo!?
Yahoo! Mail Plus - Powerful. Affordable. Sign up now.
Received on Sunday, 22 December 2002 00:41:58 UTC

This archive was generated by hypermail 2.4.0 : Friday, 17 January 2020 22:58:31 UTC