Re: Static? Dynamic?

In a previous episode Dan Connolly said...
:: 
:: John Martin wrote:
:: > 
:: > This would be regrettable. Sadly, I dont know of many web server
:: > implementations which give information providers the choice to use / set
:: > Cache-Control or Expires settings.
:: > 
[..]
:: 
:: Hmm... as an administrator, what I'd want to do is set
:: the TTL and have the server compute
:: expires = last-modified + ttl on each request.

one at a time.. 

* apache which is 56% of the web servers (netcraft 06/99) controls
   this through use of mod_expires (which is included with apache but
   not compiled in by default) and the ExpiresActive and
   Expires[ByType|Default] directives which let you set
   expires/cache-control either as an explicit amount of time or as a
   delta from modification (just as dan wish-listed)..

* Microsoft IIS which is 22% of web servers (netcraft 06/99)
   which has a section in the Internet Services Management Console
   called HTTP headers with a section entitled 'enable content
   expiration'.. which  lets you set cache-control for now, in X
   seconds in the future, or at a specific point in time

* Netscape which is 7% of web servers
   web based admin tool has a page entitled cache-control directives
   that lets you set no-cache and must-revalidate or a max-age header.

Together that's 86% of the servers on the net.. the problem seems to
be with content providers/admins who don't see it as an issue or
understand it.. indeed, I'm constantly running around here encouraging
people doing contract web development and hosting to consider the
implications of this and that a web transaction is not defined as
end-2-end.. there's a bigger picture there. as a result, in some of
our busiest deployments we have the appropriate controls set, but its
never part of the standard mindset and indeed is not a default value
in any of the above web servers..

just for giggles, I got the headers (HTTP/1.1 HEAD request) for images
off of the home pages of the organizations of John (@netapp), Dan (at
@w3) and myself (@AppliedTheory).. realistically, all of these
organizations should know enough to deal with top level *images* as
pretty darn static things... (and in turn, I bet its fair to say that
none of us as individuals are responsible for the web server
administration of these particular pages)

http://validator.w3.org/images/vh40.gif (Server: Apache/1.3.6 (Unix) PHP/3.0.9)
http://www.netapp.com/images/logondmp-home2.gif (Server: Apache/1.3.4 (Unix))
http://www.appliedtheory.com/images/logoBanner.gif (Server: Netscape-Enterprise/3.0L)

Not one of them had an Expires or cache-control header set.. all had
last modified (which is generated by default for static content on all
those servers).

I don't think this is an engineering problem anymore.. it's not even a
product/interface problem anymore.. it's a sociological and
educational problem.. people just don't see the benefit and don't set
the meta info (evidenced very anecdotally above).. and the caches try
and use heuristics to get around the lack of info (like ignoring URLs
with ? in them, even if they've got expires set.. or assuming
something that hasn't been changed in weeks is fresh for at least a
day) and then users who get frustrated with the bizarre cache
semantics (because the caches are guessing, because the content
providers are being lame) start trying to run around the caches
because they hurt them instead of helping them.. and then that leads
to 'transparent' caches which sound nice but really have as big of a
policing component to them as they do a convenience component, and
that does crush the end-2-end components of individual HTTP hops
causing a boatload of problems that don't have to be part of cache
heirarchies but get associated with them anyhow, and this perception
feeds the ugly monster all over again..

all in all its a big mess, and it's got me mighty frustrated because
I'm a huge believer in the basic tennents of the systema and that it's
the only way to really make this scale to things much bigger than
page-ready content.

-Pat

--
Patrick R. McManus - AppliedTheory Communications  -	Software Engineering
http://pat.appliedtheory.com/~mcmanus			Lead Developer
mcmanus@AppliedTheory.com	'Prince of Pollywood'	Standards, today!
*** - You Kill Nostalgia, Xenophobic Fears. It's Now or Neverland. - ***

Received on Wednesday, 30 June 1999 12:40:01 UTC