Re: Document Indexing -- How to index Dynamic Content?

Walter Ian Kaye (
Wed, 6 Nov 1996 21:26:34 -0800

Message-Id: <v03007803aea7048509bf@[]>
In-Reply-To: <>
Date: Wed, 6 Nov 1996 21:26:34 -0800
From: Walter Ian Kaye <>
Subject: Re: Document Indexing -- How to index Dynamic Content?

At 6:44p -0500 11/06/96, Ian Graham wrote:
>As we all know, many HTML documents are generated dynamically. To
>indicate this fact, most HTTP servers omit the Last-modified: HTTP
>response header field when returning dynamically generated content.
>This is reasonable, but also very crude, as often the bulk of a
>'dynamic' document does not vary, and actually has a well defined
>last-modification date, with only a small portion being varied
>on a regular basis. For example, my organization uses parsed
>HTML (*.shtml) for many home pages -- the parsing simply introduces
>a few lines of 'news of the day' text, with the majority of the
>document being invariant directory-like information for the site.
>We would like this page to be indexed by web robots, as they
>represent useful indices for the site.

How about a separate HTML page containing keywords (for indexing) and
links to your dynamic pages? Call it "botwords.html" or something... :)

Also, check out MCF (Meta-Content Format) info at:


    Walter Ian Kaye <>     Programmer - Excel, AppleScript,
          Mountain View, CA                         ProTERM, FoxPro, HTML     Musician - Guitarist, Songwriter