Hi, I tried sending this message to webmaster@msn.com and abuse@msn.com but somehow your mail system mangled that into webmaster_msn@css.one.microsoft.com which bounced. Could you please tell me the email address(es) I can use to report msnbot abuse? (see bug report, attached) -- Gerald Oskoboiny <gerald@impressive.net> http://impressive.net/people/gerald/
attached mail follows:
The original message was received at Fri, 19 Aug 2005 15:42:19 -0700 (PDT) from root@localhost ----- The following addresses had permanent fatal errors ----- webmaster_msn@css.one.microsoft.com (reason: 550 5.7.1 Requested action not taken: mailbox not available) ----- Transcript of session follows ----- .... while talking to maila.microsoft.com.: >>> DATA <<< 550 5.7.1 Requested action not taken: mailbox not available 550 5.1.1 webmaster_msn@css.one.microsoft.com... User unknown <<< 503 5.5.2 Need Rcpt command.
attached mail follows:
msnbot has been doing dumb things on my site, hitting it too
frequently (up to 25+ requests per second) and requesting the
same files over and over again.
It retrieved some files over 1000 times each, though they are
simple static files whose content does not change, with an
explicit expiry time of one day returned in the HTTP headers:
HEAD http://impressive.net/people/gerald/2003/02/19/14-05-14.rdf
200 OK
Cache-Control: max-age=86400
Connection: close
Date: Fri, 19 Aug 2005 20:37:57 GMT
Accept-Ranges: bytes
ETag: "1cb390-5e2-3e5a65fb"
Server: Apache/1.3.33 (Debian GNU/Linux) PHP/4.3.10-15 mod_ssl/2.8.22 OpenSSL/0.9.7d
Content-Length: 1506
Content-Type: text/xml
Expires: Sat, 20 Aug 2005 20:37:57 GMT
Last-Modified: Mon, 24 Feb 2003 18:35:39 GMT
Please fix your bot to be better behaved.
sample log entries:
requesting the same file over and over:
65.54.188.15 - - [19/Aug/2005:16:26:47 -0400] "GET /people/gerald/2003/02/19/14-05-14.rdf HTTP/1.0" 200 1506 "-" "msnbot/1.0 (+http://search.msn.com/msnbot.htm)" 0 impressive.net
65.54.188.15 - - [19/Aug/2005:16:30:58 -0400] "GET /people/gerald/2003/02/19/14-05-14.rdf HTTP/1.0" 200 1506 "-" "msnbot/1.0 (+http://search.msn.com/msnbot.htm)" 0 impressive.net
65.54.188.15 - - [19/Aug/2005:16:34:17 -0400] "GET /people/gerald/2003/02/19/14-05-14.rdf HTTP/1.0" 200 1506 "-" "msnbot/1.0 (+http://search.msn.com/msnbot.htm)" 0 impressive.net
65.54.188.15 - - [19/Aug/2005:16:36:51 -0400] "GET /people/gerald/2003/02/19/14-05-14.rdf HTTP/1.0" 200 1506 "-" "msnbot/1.0 (+http://search.msn.com/msnbot.htm)" 0 impressive.net
65.54.188.15 - - [19/Aug/2005:16:40:42 -0400] "GET /people/gerald/2003/02/19/14-05-14.rdf HTTP/1.0" 200 1506 "-" "msnbot/1.0 (+http://search.msn.com/msnbot.htm)" 0 impressive.net
(it has requested that URI over 1030 times in the past two days)
hitting my site too frequently:
65.54.188.15 - - [19/Aug/2005:04:10:38 -0400] "GET /people/gerald/2001/11/06/22-08-44.rdf HTTP/1.0" 200 1410 "-" "msnbot/1.0 (+http://search.msn.com/msnbot.htm)" 0 impressive.net
65.54.188.15 - - [19/Aug/2005:04:10:38 -0400] "GET /people/gerald/2001/11/03/15-18-29.rdf HTTP/1.0" 200 1421 "-" "msnbot/1.0 (+http://search.msn.com/msnbot.htm)" 0 impressive.net
65.54.188.15 - - [19/Aug/2005:04:10:38 -0400] "GET /people/gerald/2001/11/03/16-29-14.rdf HTTP/1.0" 200 1365 "-" "msnbot/1.0 (+http://search.msn.com/msnbot.htm)" 0 impressive.net
65.54.188.15 - - [19/Aug/2005:04:10:38 -0400] "GET /people/gerald/2001/11/04/13-56-23.rdf HTTP/1.0" 200 1382 "-" "msnbot/1.0 (+http://search.msn.com/msnbot.htm)" 0 impressive.net
65.54.188.15 - - [19/Aug/2005:04:10:38 -0400] "GET /people/gerald/2001/11/04/13-33-49.rdf HTTP/1.0" 200 1399 "-" "msnbot/1.0 (+http://search.msn.com/msnbot.htm)" 0 impressive.net
65.54.188.15 - - [19/Aug/2005:04:10:38 -0400] "GET /people/gerald/2001/11/03/16-12-35.rdf HTTP/1.0" 200 1365 "-" "msnbot/1.0 (+http://search.msn.com/msnbot.htm)" 0 impressive.net
65.54.188.15 - - [19/Aug/2005:04:10:38 -0400] "GET /people/gerald/2001/11/03/12-58-17.rdf HTTP/1.0" 200 1435 "-" "msnbot/1.0 (+http://search.msn.com/msnbot.htm)" 0 impressive.net
65.54.188.15 - - [19/Aug/2005:04:10:38 -0400] "GET /people/gerald/2001/11/08/14-47-29.rdf HTTP/1.0" 200 1417 "-" "msnbot/1.0 (+http://search.msn.com/msnbot.htm)" 0 impressive.net
65.54.188.15 - - [19/Aug/2005:04:10:38 -0400] "GET /people/gerald/2001/11/08/07-03-20.rdf HTTP/1.0" 200 1409 "-" "msnbot/1.0 (+http://search.msn.com/msnbot.htm)" 0 impressive.net
65.54.188.15 - - [19/Aug/2005:04:10:38 -0400] "GET /people/gerald/2001/11/08/11-42-23.rdf HTTP/1.0" 200 1466 "-" "msnbot/1.0 (+http://search.msn.com/msnbot.htm)" 0 impressive.net
65.54.188.15 - - [19/Aug/2005:04:10:38 -0400] "GET /people/gerald/2001/11/08/12-43-15.rdf HTTP/1.0" 200 1412 "-" "msnbot/1.0 (+http://search.msn.com/msnbot.htm)" 0 impressive.net
65.54.188.15 - - [19/Aug/2005:04:10:38 -0400] "GET /people/gerald/2001/11/04/14-57-00.rdf HTTP/1.0" 200 1391 "-" "msnbot/1.0 (+http://search.msn.com/msnbot.htm)" 0 impressive.net
65.54.188.15 - - [19/Aug/2005:04:10:38 -0400] "GET /people/gerald/2001/11/08/14-38-18.rdf HTTP/1.0" 200 1528 "-" "msnbot/1.0 (+http://search.msn.com/msnbot.htm)" 0 impressive.net
65.54.188.15 - - [19/Aug/2005:04:10:38 -0400] "GET /people/gerald/2001/11/08/14-19-00.rdf HTTP/1.0" 200 1414 "-" "msnbot/1.0 (+http://search.msn.com/msnbot.htm)" 0 impressive.net
65.54.188.15 - - [19/Aug/2005:04:10:38 -0400] "GET /people/gerald/2004/09/29/21-02-07.rdf HTTP/1.0" 200 1497 "-" "msnbot/1.0 (+http://search.msn.com/msnbot.htm)" 0 impressive.net
65.54.188.15 - - [19/Aug/2005:04:10:38 -0400] "GET /people/gerald/2001/11/04/14-21-51.rdf HTTP/1.0" 200 1402 "-" "msnbot/1.0 (+http://search.msn.com/msnbot.htm)" 0 impressive.net
65.54.188.15 - - [19/Aug/2005:04:10:38 -0400] "GET /people/gerald/2001/11/04/12-30-04.rdf HTTP/1.0" 200 1362 "-" "msnbot/1.0 (+http://search.msn.com/msnbot.htm)" 0 impressive.net
65.54.188.15 - - [19/Aug/2005:04:10:38 -0400] "GET /people/gerald/2001/11/03/15-41-19.rdf HTTP/1.0" 200 1365 "-" "msnbot/1.0 (+http://search.msn.com/msnbot.htm)" 0 impressive.net
65.54.188.15 - - [19/Aug/2005:04:10:38 -0400] "GET /people/gerald/2001/11/08/11-21-41.rdf HTTP/1.0" 200 1406 "-" "msnbot/1.0 (+http://search.msn.com/msnbot.htm)" 0 impressive.net
65.54.188.15 - - [19/Aug/2005:04:10:38 -0400] "GET /people/gerald/2001/11/08/14-17-37.rdf HTTP/1.0" 200 1414 "-" "msnbot/1.0 (+http://search.msn.com/msnbot.htm)" 0 impressive.net
65.54.188.15 - - [19/Aug/2005:04:10:38 -0400] "GET /people/gerald/2003/02/19/14-03-46.rdf HTTP/1.0" 200 1443 "-" "msnbot/1.0 (+http://search.msn.com/msnbot.htm)" 0 impressive.net
--
Gerald Oskoboiny <gerald@impressive.net>
http://impressive.net/people/gerald/
This archive was generated by hypermail 2.2.0+W3C-0.50 : Monday, 7 July 2008 08:09:40 GMT