Active proxy 'problems'

Hello all,

It seems I have stumbled across an interesting problem:
I've developed a web-site for a customer which relies heavely on a database
with nearly 1 million records in several tables. To relief the CPU and
memory I've decided to deploy a front side proxy (proxy cache on the
webserver). Most of the pages have a life-time of 24 hours by using the
'Expires:' header.

This setup seems to work quite well, but there is one major drawback: There
seem to be active proxies at the client side. This means that more and more
sites will refresh their caches every 24 hours. This results in a huge
amount of wasted bandwith since no-one (= a person) at the client side
actually requested the document.

I've read the parts that concern caching from the RFC2616 (Hypertext
Transfer Protocol -- HTTP/1.1 ) but it seems there is no way to tell an
active proxy cache at the client side: "Do not refresh this document until
somebody actually wants it!".

Did I overlook a possible solution or is this really a new problem? 
It seems more websites are moving towards front side proxies to reduce the
load on database servers so this may become a potential problem for the
internet.

For some reason I also posted this message to the www-html list (probably
cut&paste from the example page). My apologies to anybody who has received
this message twice.

Nico Coesel

Received on Monday, 1 July 2002 07:31:09 UTC