W3C home > Mailing lists > Public > www-talk@w3.org > May to June 2009

Re: Http 403 Error for W3 DTDs

From: Ted Guild <ted@w3.org>
Date: Wed, 10 Jun 2009 18:30:30 -0400
To: Sumit Shah <Sumit.Shah@cgifederal.com>
Cc: www-talk@w3.org
Message-ID: <nnbk53jpvqx.fsf@dev-null.guilds.net>
Sumit,

We are sending HTTP 503 and the content of the response also includes a
link which expands to an article giving more background on this issue.

http://www.w3.org/blog/systeam/2008/02/08/w3c_s_excessive_dtd_traffic

In the last 16 months since writing that article we have only seen
this traffic increase and recently we are seeing surges in traffic
that we cannot keep up with, neither our automated defenses nor manual
intervention.  Increasing server capacity sees the increased capacity
just getting consumed as well.  This is rendering our site overwhelmed
and unresponsive for our working groups and the rest of the web
community.

> I was wondering if this is an isolated issue or something across the
> board. Is this something intentional that W3 has done to block DTD
> requests and is there a suggested fix for it?

About 1/4th of our DTD traffic (in the hundreds of millions/day) is from
Java so when trying to keep our site available yesterday responding 503
to this traffic was low hanging fruit.  We will be monitoring this
traffic and see when we can be less dramatic in our defenses.

We have also identified another widely distributed application
responsible for a substantial portion of this traffic, the vendor has
acknowledged the issue and is working on a resolution which we hope
will be released soon.

Many libraries have catalog or caching options and lacking that one can
get a caching proxy in front of their application making repeated DTD
requests.

-- 
Ted Guild <ted@w3.org>
W3C Systems Team
http://www.w3.org
Received on Wednesday, 10 June 2009 22:30:42 GMT

This archive was generated by hypermail 2.2.0+W3C-0.50 : Wednesday, 27 October 2010 18:14:31 GMT