W3C home > Mailing lists > Public > www-talk@w3.org > September to October 1996

my site is being attacked by Micorosoft proxy servers

From: Rob Hartill <robh@imdb.com>
Date: Thu, 19 Sep 1996 02:19:15 +0100 (BST)
Message-Id: <199609190119.CAA00536>
To: www-talk@w3.org

For weeks now my servers have been pounded by users of Microsoft
brain-dead proxy server "MS-Catapult".

This wonderful network-friendly software has a horrible bug in it
that lots of its users have managed to trigger, yet they've failed
to notice it.

Multiple sites are hitting my servers over and over and over for the
same URLs. All these requests come with the USER_AGENT MS-Catapult/0.9.
The requests have a frequency of 1-60 seconds where 3-7 URLs (usually
gifs) are requested together every N (1-60) seconds.

I've already hacked my Apache server to reject (403 Forbidden) all
requests from MS-Catapult agents, but does that stop them coming?, no!;
they're just as happy to request a hundred thousand times and be told
"forbidden" a hundred thousand times. And these numbers are real, I've
seen millions of requests from the various sites in the past few weeks.
They're non-stop 24 hours a day, 7 days a week.

I've sent multiple mail messages to Microsoft's postmaster. No 
acknowledgement. I've sent mail to postmaster/root at the proxy server
sites, again no acknowledgement, and no let up in the barrage of
requests.

Is there someone out there who can point me at the developers of
MS-Catapult?

The situation gets worse with time as more and more proxies return to
the site and get into this infinite loop. I've spent far too many days
cleaning up the mess and it's not fair on the legitimate users of the
site that access times are worsened because the proxies are using up
the resources.

Attached is a patch for Apache (1.1.1 or higher) that I now use to
block MS-Catapult. It is used by adding (to access.conf)

<Location />
  <Limit GET>
  order allow,deny
  allow from all
  deny from user-agents MS-Catapult
  </Limit>
</Location>

it doesn't stop the requests flooding in, but it does stop the proxies
from grabbing larger files or running scripts.

If you are using MS-Catapult, please consider switching it off until
Microsoft acknowledge the problem and offer a fix. If you don't, your
proxy will probably waste more network resources that it is supposed to
be saving.

-- 
Rob Hartill (robh@imdb.com)    
http://www.imdb.com/  ... why wait for a clear night to see the stars?.


       ========================================================

*** mod_access.c.orig	Tue Sep 10 12:11:48 1996
--- mod_access.c	Tue Sep 10 13:11:34 1996
***************
*** 178,183 ****
--- 178,194 ----
      for (i = 0; i < a->nelts; ++i) {
          if (!(mmask & ap[i].limited))
  	    continue;
+         if (ap[i].from && !strcmp(ap[i].from, "user-agents")) {
+            char * this_agent = table_get(r->headers_in, "User-Agent");
+ 	   int j;
+ 
+            if (!this_agent) return 0;
+ 
+            for (j = i+1; j < a->nelts; ++j) {
+               if (strstr(this_agent, ap[j].from)) return 1;
+            }
+            return 0;
+         }
  	if (!strcmp (ap[i].from, "all"))
  	    return 1;
  	if (!gothost)
Received on Wednesday, 18 September 1996 21:21:44 GMT

This archive was generated by hypermail 2.2.0+W3C-0.50 : Wednesday, 27 October 2010 18:14:19 GMT