Re: Changes to Content Negotiation, Entity Tags, and If-*

Koen asked me to explain why the draft 03 sections on negotiation
were incorrect descriptions of what can be done in terms of negotiation
(and when negotiation applies).  So, I'll do that first, using only
my diffs as reference.  I'll reply to Koen's comments later.

> ==========================================================================
> *** draft-03.txt	Fri May 24 23:39:00 1996
> --- cneg.txt	Sat May 25 09:07:17 1996
> ***************
> *** 476,501 ****
>   
>   resource
>     A network data object or service that can be identified by a URI
> !   (section 7.2).  At any point in time, a resource may be either a
> !   plain resource, which corresponds to only one possible
> !   representation, or a generic resource.
> 
> - generic resource
> -   A resource that is a set of closely related representations of the
> -   same document, form, applet, etc. A generic resource is always
> -   identified by a URI. The individual representations may each be
> -   identified by a unique URI, or by the combination of the generic
> -   resource's URI and a variant-ID, or by the combination of the generic
> -   resource's URI and some "content-negotiation" mechanism.  In this
> -   case, other URIs may exist which identify a resource more
> -   specifically.
>   
> - plain resource
> -   A resource that is not a generic resource.  A plain resource is
> -   always identified by a URI.
> - 
> - 
> --- 476,484 ----
>   
>   resource
>     A network data object or service that can be identified by a URI
> !   (section 7.2).
>   

REASON: It is not necessary to discuss the nature of the resource. 
In order for caching to work correctly, the cache only needs to know what
to do when it receives the Vary header field in a response.  This
difference in description allows any response (including errors) to be
negotiated, whereas draft 03 mistakenly disallows that capability.
This is what Henrik was trying to explain in Paris, and it wasn't until
I read through the negotiation sections again that I understood why.

> ***************
> *** 510,543 ****
>     Entity-Header fields and content in the form of an Entity-Body, as
>     described in section 11.
>   
> ! resource entity
> !   A specific representation, rendition, encoding, or presentation of a
> !   network data object or service, either a plain resource or a specific
> !   member of a generic resource.  A resource entity might be identified
> !   by a URI, or by the combination of a URI and a variant-ID, or by the
> !   combination of a URI and some other mechanism. An plain resource MUST
> !   be bound to a single resource entity at any instant in time.
>   
> - variant
> -   A resource entity that is a member of at least one generic resource.
> -   Sometimes called a resource variant.  Note that the set of variants
> -   of a generic resource may change over time as well.
> - 
> - content negotiation
> -   The mechanism for selecting the appropriate variant of a generic
> -   resource when servicing a request, as described in section 15.
> - 
> - entity tag
> -   An opaque string associated with an entity and used to distinguish it
> -   from other entities of the requested resource .  A "strong entity
> -   tag" is one that may be shared by two entities of a resource only if
> -   they are equivalent by octet equality.  A "weak entity tag" is one
> -   that may be shared by two entities of a resource if they are
> -   equivalent and could be substituted for each other with no
> -   significant change in semantics.  A given entity tag value may be
> -   used for entities obtained by requests on different URIs without
> -   implying anything about the equivalence of these entities.
> - 
>   client
>     An application program that establishes connections for the purpose
>     of sending requests.
> --- 493,504 ----
>     Entity-Header fields and content in the form of an Entity-Body, as
>     described in section 11.
>   
> ! representation
> !   An entity included with a response that is subject to content
> !   negotiation, as described in section 15.  There may exist multiple
> !   representations associated with a particular response status of a
> !   requested resource.
>   
>   client
>     An application program that establishes connections for the purpose
>     of sending requests.

REASON: Discussing what Content Negotiation means is the purpose of that
section, and does not belong in terminology.  Likewise, entity tag has its
own section.  I added "representation" because it is a better term than
variant for the case where only one exists.  This was also the term
commonly used when talking about these things (even on the conneg list).

> ***************
> *** 1681,1768 ****
>   
>   
>   7.11 Entity Tags
> - Entity tags are quoted strings whose internal structure is not visible
> - to clients or caches. Entity tags are used as cache validators in
> - HTTP/1.1.
>   
> !       entity-tag = strong-entity-tag | weak-entity-tag
> !                               | null-entity-tag
> !       strong-entity-tag = quoted-string
> !       weak-entity-tag = quoted-string "/W"
> !       null-entity-tag = <"> <">
>   
> !   Note that the "/W" tag is considered part of a weak entity tag; it
> !   MUST NOT be removed by any cache or client.
>   
> ! There are two comparison functions on  validators:
>   
> !   .  The strong comparison function: in order to be considered equal,
> !      both validators must be identical in every way, and neither may be
> !      weak.
>   
> -   .  The weak comparison function: in order to be considered equal, both
> -      validators must be identical in every way, except for the presence
> -      or absence of a "weak" tag.
>   The weak comparison function MAY be used for simple (non-subrange) GET
> ! requests. The strong comparison function MUST be used in all other
>   cases.
>   
> ! The null validator is a special value, defined as never matching the
> ! current validator of an existing resource entity, and always matching
> ! the "current" validator of a resource entity  that does not exist.
>   
>   
> - 7.12 Variant IDs
> - A cache stores instances of resource entities, not instances of generic
> - resources per se. Therefore, the URI of a generic resource is not
> - sufficient for use as an identifier for a specific resource entity. In
> - certain interactions between a cache and an origin server, it is
> - convenient to encode that identifier using a more   compact
> - representation than the full set of selecting request headers (which may
> - not even be possible if the selection criteria are not known to the
> - cache).
> - 
> - For these reasons, the HTTP protocol provides an optional mechanism for
> - identifying a specific entity source of a generic resource,  called a
> - variant-ID.
> - 
> - Variant-IDs are used to identify specific variants of a generic
> - resource; see section 16.5.3 for how they are used.
> - 
> -       variant-id = quoted-string
> - 
> - Variant-IDs are compared using string octet-equality; case is
> - significant.
> - 
> - All responses from generic resources SHOULD include variant-IDs.  If
> - these are not present, the resource author can expect caches to
> - correctly handle requests on the generic resource, but cannot expect the
> - caching to be efficient.
> - 
> - 
> - 
> - 
> - 7.13 Variant Sets
> - Validator sets are used for doing conditional retrievals on generic
> - resources; see section 16.5.3.
> - 
> -       variant-set = 1#variant-set-item
> -       variant-set-item = opaque-validator ";" variant-id
> - 
> --- 1642,1683 ----
>   
>   
>   7.11 Entity Tags
>   
> ! Entity tags are used for comparing two or more entities from the same
> ! requested resource.  An entity tag consists of a quoted string, whose
> ! internal structure is opaque to clients, possibly prefixed by a
> ! weakness indicator.
>   
> !       entity-tag = [ weak ] opaque-tag
>   
> !       weak       = "W/"
> !       opaque-tag = quoted-string
>   
> ! A "strong entity tag" may be shared by two entities of a resource only
> ! if they are equivalent by octet equality. 
>   
> ! A "weak entity tag", indicated by the "W/" prefix, may be shared by
> ! two entities of a resource only if they are equivalent and could be
> ! substituted for each other with no significant change in semantics.
> ! A weak entity tag can only be used for weak comparison.
>   
> + There are two comparison functions on entity tags:
>   
> +   .  The strong comparison function: in order to be considered equal,
> +      both entity tags must be identical, and neither may be weak.
>   
> +   .  The weak comparison function: in order to be considered equal, 
> +      the opaque-tags of both entity tags must be identical.
>   
>   The weak comparison function MAY be used for simple (non-subrange) GET
> ! or HEAD requests. The strong comparison function MUST be used in all other
>   cases.
>   
> ! A given entity tag value may be used for entities obtained by requests
> ! on different URIs without implying anything about the equivalence of
> ! those entities.
>   
REASON: These changes were agreed to by the editorial group after
extensive discussion in Paris and in Palo Alto.  We went through all
known scenarios of use and discovered that the only advantage held by
variant-id's was that they allowed a cache to delete old entries of
the same variant earlier than a normal cache flush, and even then it
would only be useful in those cases where Content-Location was not
provided.  Since variant-id's were added to draft 02/03 without any
discussion on the WG list, there was no justification for keeping them
until after further discussion on removal.  They never had consensus
from any subgroup.  The weakness "W" was moved from a suffix to a prefix
at my insistence in order to make it easier for a cache to check.

The only thing I did independently here is to move the four-five areas
where entity tag was defined into the single place it belonged.

> ***************
> *** 2731,2744 ****
>   
>   
>   12.3.1.1 300 Multiple Choices
> - This status code is reserved for future use by a planned content
> - negotiation mechanism.  HTTP/1.1 user agents receiving a 300 response
> - which includes a Location header field can treat this response as they
> - would treat a 303 (See Other) response.  If no Location header field is
> - included, the appropriate action is to display the entity enclosed in
> - the response to the user.
>   
>   
>   12.3.1.2 301 Moved Permanently
>   The requested resource has been assigned a new permanent URI and any
>   future references to this resource SHOULD be done using one of the
> --- 2646,2671 ----
>   
>   
>   12.3.1.1 300 Multiple Choices
>   
> + The requested resource is available in one or more representations,
> + each with their own specific location, and agent-driven negotiation
> + information (section 15) is being provided so that the user (or user
> + agent) can select a preferred representation.
>   
> + Unless it was a HEAD request, the response SHOULD include an entity
> + containing a list of resource characteristics and locations from which
> + the user or user agent can choose the one most appropriate. The entity
> + format is specified by the media type given in the Content-Type header
> + field. Depending upon the format and the capabilities of the user agent,
> + selection of the most appropriate choice may be performed automatically.
> + However, this specification does not define any standard format for
> + such automatic selection.
> + 
> + If the server has a preferred choice of representation, it SHOULD
> + include the specific URL for that representation in the Location field;
> + user agents MAY use the Location field value for automatic redirection.
> + This response is cachable unless indicated otherwise.
> + 
>   12.3.1.2 301 Moved Permanently
>   The requested resource has been assigned a new permanent URI and any
>   future references to this resource SHOULD be done using one of the

REASON: All status codes key off the first digit and the specification
requires that all *unrecognized* status codes be treated as the x00
status of that class.  It is totally unacceptable for HTTP/1.1 not to
fully define the 300 status code.  You will note that there is nothing
in this definition that would prohibit (or hamper) future negotiation
mechanisms.  In fact, without this default interpretation as a fall-back
for simple agents, we would never be able to deploy any new negotiation
mechanisms.

> ***************
> *** 2924,2929 ****
> --- 2851,2865 ----
>   response entities which have content characteristics not acceptable
>   according to the accept headers sent in the request.
>   
> + Unless it was a HEAD request, the response SHOULD include an entity
> + containing a list of available entity characteristics and locations
> + from which the user or user agent can choose the one most appropriate.
> + The entity format is specified by the media type given in the
> + Content-Type header field. Depending upon the format and the
> + capabilities of the user agent, selection of the most appropriate
> + choice may be performed automatically.  However, this specification
> + does not define any standard format for such automatic selection.
> + 
>   HTTP/1.1 servers are allowed to return responses which are not
>   acceptable according to the accept headers sent in the request. In some
>   cases, this may even be preferable to sending a 406 response.  User

REASON: [this is for the 406 code] ALL error codes are supposed to include
an entity describing the reason for the error response and (hopefully)
some workaround for the user.  That is how HTTP was designed and how
it will remain if we want older applications to be useful in the face of
future extensions.  There was no reason for draft 02/03 to remove it.

> ***************
> *** 2932,2938 ****
>   agents SHOULD interrupt the receipt of the response if doing so would
>   save network resources.  If it is unknown whether an incoming response
>   would be acceptable, a user agent SHOULD temporarily stop receipt of
> ! more data and query the user for a decision on furtheractions.
>   
>   
>   12.4.1.8 407 Proxy Authentication Required
> --- 2868,2874 ----
>   agents SHOULD interrupt the receipt of the response if doing so would
>   save network resources.  If it is unknown whether an incoming response
>   would be acceptable, a user agent SHOULD temporarily stop receipt of
> ! more data and query the user for a decision on further action.
>   
>   
>   12.4.1.8 407 Proxy Authentication Required

REASON: Typo

The following I will have to explain piece-by-piece:

> ***************
> *** 3656,3724 ****
>   
>   
>   15 Content Negotiation
> - A generic resource has multiple entities associated with it, all of
> - which are representations of the content of the resource.  Content
> - negotiation is the process of selecting the best representation when a
> - GET or HEAD request is made on the generic resource.  HTTP/1.1 has
> - provisions for two kinds of content negotiation: opaque negotiation and
> - transparent negotiation.

All three sentences above are dead wrong.  Content negotiation may apply
to any response, not just okay responses to GET and HEAD.  That means 
PUT, POST, etc., responses can be negotiated, and error responses can also
be negotiated.  These facts have been demonstrated in current practice.

It is also inaccurate to describe HTTP negotiation as being "opaque" vs
"transparent" -- that has never been the case.  Draft 01 correctly
described it as "preemptive" vs "reactive", and that was the consensus
of the conneg group (I just reviewed the mailing list).  I changed the
names to "server-driven" and "agent-driven" because many people asked
me to come up with names that would be more intuitive to the reader.
The important thing, however, is where the selection decision is made,
not whether or not the selection algorithm is a standard, or even
automated (e.g., "click here for the postscript version" is a degenerate
example of agent-driven negotiation in current practice).

Transparent negotiation is, as I described it, a combination of the
two above kinds of negotiation over a restricted set of common attributes.

> ! With opaque negotiation, the selection of the best representation is
> ! done by an algorithm located at the origin server, and unknown to the
> ! proxies and user agents involved.  Selection is based on the contents of
> ! particular header fields in the request message, or on other information
> ! pertaining to the request, like the network address of the sending
> ! client.  A typical example of opaque negotiation would be the selection
> ! of a text/html response in a particular language based on the contents
> ! of the Accept-Language request header field.  A disadvantage of opaque
> ! negotiation is that the request headers may not always contain enough
> ! information to allow for selection.  If the Accept header
>   
> !         Accept: text/*: q=0.3, text/html, */*: q=0.5
>   
> ! is sent in a request on a generic resource which has a video/mpeg and a
> ! video/quicktime representation, the selection algorithm in the origin
> ! server will either have to make a default choice, or return an error
> ! response which allows the user to decide on further actions.
>   
> ! With transparent negotiation, the selection of the best representation
> ! is done by a distributed algorithm which can perform computation steps
> ! in the origin server, in proxies, or in the user agent.  Transparent
> ! negotiation guarantees that, if the user agent supports the transparent
> ! negotiation algorithm and is correctly configured, the request will
> ! always correctly yield either the video/mpeg representation, the
> ! video/quicktime representation, or an error message indicating that the
> ! resource cannot be displayed by the user agent.
>   
>   
> ! 15.1  Negotiation Facilities Defined in this Specification
> ! This specification defines all protocol facilities for opaque
> ! negotiation, but does not define the distributed algorithm for
> ! transparent negotiation.  This specification only defines the basic
> ! facilities (Vary, Alternates, Accept) in the core protocol allowing
> ! requests on transparently negotiated resources to be correctly handled
> ! by HTTP/1.1 caches.  All other information about transparent content
> ! negotiation is found in a separate document[29].

The above paragraph makes a normative reference to a nonexistant document.
Proposed standards cannot do that.  Moreover, it incorrectly describes
the Vary behavior as being a lesser cousin to some future Alternates,
which it is not once the specification of Vary is correctly described (below).

> ! If a generic resource is opaquely negotiated, successful responses to
> ! requests on the resource will always include a Vary header.  If a
> ! generic resource is transparently negotiated, successful responses to
> ! requests on the resource will always include an Alternates header.  If a
> ! successful response contains an Alternates header, it will also always
> ! contain a Content-Location header.  A future specification may allow a
> ! combination of opaque and transparent negotiation that would lead to the
> ! inclusion of both a Vary header and an Alternates header in a response.

I made myself perfectly clear on at least six different occasions, on
phone and in person, that Vary and Alternates are orthogonal concepts and
must be defined that way.  The same conclusion can be seen in the 
consensus represented in the conneg group mailing list. Draft 03 was
completely wrong in its description of the consensus on conneg and of
the protocol in question.  See
<http://www.organic.com/public/conneg/mail/0150.html> for reference.

I finally gave up on asking and did the appropriate wording myself.

> --- 3592,3736 ----
>   
>   
>   15 Content Negotiation
>   
> ! Most HTTP responses include an entity which contains information
> ! for interpretation by a human user. Naturally, it is desirable to
> ! supply the user with the "best available" entity corresponding to the
> ! request. Unfortunately for servers and caches, not all users have the
> ! same preferences for what is "best", and not all user agents are
> ! equally capable of rendering all entity types. For that reason, HTTP
> ! supports various mechanisms for "content negotiation" -- the process
> ! of selecting the best representation of a resource for a given response
> ! when there are multiple representations available.
>   
> !    Note: This is not called "format negotiation" because the alternate
> !    representations may be of the same media type, but use different
> !    capabilities of that type, be in different languages, etc.
>   
> ! Any response containing an entity-body MAY be subject to negotiation,
> ! including error responses.
>   
> ! There are two kinds of content negotiation which are possible in HTTP:
> ! server-driven negotiation (also known as preemptive or opaque negotiation)
> ! and agent-driven negotiation (also known as reactive negotiation).
> ! These two kinds of negotiation are orthogonal and thus may be used
> ! separately or in combination without affecting the interpretation of
> ! a response. One method of combination, referred to as transparent
> ! negotiation, occurs when a cache uses the agent-driven negotiation
> ! information provided by the origin server in order to provide
> ! server-driven negotiation for subsequent requests.
>   
> + 15.1 Server-driven Negotiation
>   
> + With server-driven negotiation, the selection of the best representation
> + for a response is done by an algorithm located at the origin server and
> + unknown to the client(s) receiving the response.  Selection is based on
> + the available representations of the response (the dimensions over which
> + it can vary) and either the contents of particular header fields in the
> + request message or on other information pertaining to the request
> + (such as the network address of the client).
>   
> + Server-driven negotiation is advantageous when the algorithm for
> + selecting from among the available representations is difficult to
> + describe to the user agent, or when the server desires to send its
> + "best guess" to the client along with the first response (hoping to
> + avoid the round-trip delay of a subsequent request if the "best guess"
> + is good enough for the user).  In order to improve the server's guess,
> + the user agent MAY include request header fields (Accept, Accept-Language,
> + Accept-Encoding, etc.) which describe its preferences for such a
> + response.
>   
> ! Server-driven negotiation has several disadvantages. First, it is
> ! impossible for the server to accurately determine what might be "best"
> ! for any given user, since that would require complete knowledge of
> ! both the capabilities of the user agent and the intended use for the
> ! response (e.g., does the user want to view it on screen or print it
> ! on paper?). Second, having the user describe its capabilities in every
> ! request is both horrendously inefficient (given that only a small
> ! percentage of responses have multiple representations) and a potential
> ! violation of the user's privacy. Third, it significantly complicates
> ! the implementation of an origin server and the algorithms for generating
> ! responses to a request. Finally, it could interfere with a public
> ! cache's ability to use the same response for multiple user's requests.
>   
> + HTTP/1.1 includes the following request-header fields for enabling
> + server-driven negotiation through description of user agent capabilities
> + and user preferences: Accept (section 18.1), Accept-Charset (section 18.2),
> + Accept-Encoding (section 18.3), Accept-Language (section 18.4), and
> + User-Agent (section 18.45). However, an origin server is not limited to
> + these dimensions and MAY vary the response based on any aspect of the
> + request, including information outside the request-header fields or
> + within extension header fields not defined by this specification.
>   
> ! HTTP/1.1 origin servers MUST include an appropriate Vary header field
> ! (section 18.46) in any response based on server-driven negotiation.
> ! The Vary header field describes the dimensions over which the response
> ! might vary (i.e., the dimensions over which the origin server picks its
> ! "best guess" response from multiple representations).
>   
> + HTTP/1.1 public caches MUST recognize the Vary header field when it
> + is included in a response and obey the requirements described in
> + section 16.XX on caching negotiated responses.
>   
> ! 15.2 Agent-driven Negotiation
>   
> ! With agent-driven negotiation, selection of the best representation
> ! for a response is performed by the user agent after receiving an initial
> ! response from the origin server. Selection is based on a list of the
> ! available representations of the response included within the header
> ! fields (Alternates, appendix 23.2.5.1) or entity-body of the initial
> ! response, with each alternative identified by its own URI. Selection
> ! from among the alternatives may be performed automatically (if the
> ! user agent is capable of doing so) or manually by the user selecting
> ! from a generated (possibly hypertext) menu.
>   
> + Agent-driven negotiation is advantageous when the response would vary
> + over commonly-used dimensions (such as type, language, or encoding),
> + when the origin server is unable to determine a user agent's capabilities
> + from examining the request, and at any time when public caches are used
> + to distribute server load and reduce network usage.
>   
> ! Agent-driven negotiation suffers from the disadvantage of needing a
> ! second request to obtain the best alternate representation.  This second
> ! request is only efficient when hierarchical caching is used.  In addition,
> ! this specification does not define any mechanism for supporting
> ! automatic selection, though it also does not prevent any such mechanism
> ! from being developed as an extension and used within HTTP/1.1.
> ! 
> ! HTTP/1.1 defines the 300 (multiple choices) and 406 (not acceptable)
> ! status codes for enabling agent-driven negotiation when the server is
> ! unwilling or unable to provide a varying response using server-driven
> ! negotiation.
> ! 
> ! 15.3 Transparent Negotiation
> ! 
> ! Transparent negotiation is a combination of both server-driven and
> ! agent-driven negotiation.  When a cache is supplied with an automated
> ! form of the list of available representations of the response
> ! (as in agent-driven negotiation) and the dimensions of variance are
> ! completely understood by the cache, then the cache becomes capable of
> ! performing server-driven negotiation on behalf of the origin server
> ! for subsequent requests on that resource.
> ! 
> ! Transparent negotiation has the advantage of distributing the
> ! negotiation work that would otherwise be required of the origin server
> ! and also removing the second request delay of agent-driven negotiation
> ! when the cache is able to correctly guess the right response and
> ! already has that response cached.
> ! 
> ! A cache performing transparent negotiation MUST include the agent-driven
> ! negotiation information along with the response, and MUST add a Vary
> ! header field to the response (defining the dimensions of its variance)
> ! if a Vary field was not already assigned by the origin server.
> ! 
> ! These requirements apply to HTTP/1.1 applications even though this
> ! specification does not include a means for accomplishing transparent
> ! negotiation, since an understanding of these requirements is a
> ! necessary prerequisite for any future implementation of these features.
> ! 
> ***************
> *** 4204,4220 ****
>         response_is_fresh = (freshness_lifetime > current_age)
>   
>   
> - 16.2.6 Scope of Expiration
> - HTTP/1.1's expiration model is that as soon as any variant of a URI
> - becomes stale, all variants becomes stale as well.  Thus, "freshness"
> - applies to all the variants of URI, rather than any particular variant.
> - Dates and expires etc. apply to any cached variant that a proxy might
> - have with a URI and not just the one particular entity.
> - 
> - Editor's note: This restriction may be dropped in the next draft; there
> - are still discussions about whether this restriction is needed.
> - 
> - 
>   16.2.7 Disambiguating Expiration Values
>   Because expiration values are assigned optimistically, it is possible
>   that two caches may contain fresh values for the same resource that are
> --- 4216,4221 ----

REASON: The restriction was not needed and, more importantly, was redundant
to the normal behavior of a cache upon receipt of any response.

> ***************
> *** 4338,4378 ****
>   
>   
>   16.3.1 Last-modified Dates
> - In HTTP/1.0, the only cache validator is the Last-Modified time carried
> - by a response. Clients validate entities using the If-Modified-Since
> - header. In simple terms, a cache entry is considered to be valid if the
> - actual resource entity has not been modified since the original response
> - was generated.

This is wrong because all aspects of caching are independent of the protocol
version. There is nothing preventing an HTTP/1.0 client or server from
implementing cache-control, Etag, etc.

>   16.3.2 Entity Tags
> - HTTP/1.1 introduces the possibility of using an "opaque" validator,
> - called an "entity tag," for situations where the Last-Modified date is
> - not appropriate. This may include server implementations where it is not
> - convenient to store modification dates, or where the one-second
> - resolution of HTTP date values is insufficient, or where the origin
> - server wishes to avoid certain paradoxes that may arise from the use of
> - modification dates.
>   
> ! An entity tag is simply a string of octets whose internal structure is
> ! not known to clients or caches. Caches store entity tags and return them
> ! when making conditional requests. Also, when a cache receives a
> ! conditional request for a resource for which it has a fresh cache
> ! entry,
> ! it may compare entity tags using strict octet-equality. Otherwise,
> ! entity tags have no semantic value to clients or caches.
>   
> ! To preserve compatibility with HTTP/1.0 clients and caches, and because
> ! the Last-Modified date may be useful for purposes other than cache
> ! validation, HTTP/1.1 servers SHOULD send Last-Modified whenever
> ! feasible.
>   
> - The headers used to convey entity tags are described in sections Error!
> - Reference source not found., Error! Reference source not found., 18.26,
> - and 18.46.
> - 
> --- 4339,4362 ----
>   
>   
>   16.3.1 Last-modified Dates
>   
> + The Last-Modified entity-header field value is often used as a cache
> + validator. In simple terms, a cache entry is considered to be valid if the
> + resource has not been modified since the Last-Modified value.
>   
>   16.3.2 Entity Tags
>   
> ! The Etag entity-header field value, an entity tag, provides for an
> ! "opaque" cache validator.  This may allow more reliable validation in
> ! situations where it is inconvenient to store modification dates, where
> ! the one-second resolution of HTTP date values is not sufficient, or
> ! where the origin server wishes to avoid certain paradoxes that may
> ! arise from the use of modification dates.
>   
> ! Entity tags are described in section 7.11.
>   

REASON: This is just the result of moving all definitions associated
with entity tag into the entity tag section where they belong.  We could
probably reduce this even more, but I need to wait for a new draft before
I can do an accurate slice through the Caching section.

> ***************
> *** 4543,4549 ****
>   
>     .  If an entity tag has been provided by the origin server, MUST use
>        that entity tag in any cache-conditional request (using If-Match or
> !      If-NoneMatch).
>     .  If only a Last-Modified value has been provided by the origin
>        server, SHOULD use that value in non-subrange cache-conditional
>        requests (using If-Modified-Since).
> --- 4527,4533 ----
>   
>     .  If an entity tag has been provided by the origin server, MUST use
>        that entity tag in any cache-conditional request (using If-Match or
> !      If-None-Match).
>     .  If only a Last-Modified value has been provided by the origin
>        server, SHOULD use that value in non-subrange cache-conditional
>        requests (using If-Modified-Since).

REASON: Typo.

> ***************
> *** 4716,4818 ****
>   missing), and must discard the other partial information.
>   
>   
> ! 16.5 Caching and Generic Resources
> ! Generic resources interacts with caching in several ways:
>   
> !   .  A generic resource (one subject to content negotiation) may be
> !      bound to more than one entity. Each of these entities is called a
> !      "variant" of the resource.
> !   .  The request-URI may be only one part of the cache key.
>   
> ! 16.5.1 Vary Header Use
> ! Origin servers may respond to requests for generic resources use the
> ! Vary header (see section 18.46 for a full description) to inform the
> ! cache which header fields of the request were used to select the variant
> ! returned in the response. A cache can use that response to reply to a
> ! subsequent request only if the two requests not only specify the same
> ! URI, but also have the same value for all headers specified in the Vary
> ! response-header.
>   
> ! The Vary header may also inform the cache that the variant was selected
> ! using criteria not limited to the request headers; in this case, the
> ! response MUST NOT be used in a reply to a subsequent request except if
> ! the cache relays the new request to the origin server in a conditional
> ! request, and the origin server responds with 304 (Not Modified) and
> ! includes the same variant-ID (see 13.8.3).
>   
>   
> - 16.5.2 Alternates Header Use
> - The Alternates header is present in the HTTP/1.1 to enable caching of
> - entities from the planned content negotiation facilities. If a cache
> - receives an Alternates header in a response from the origin server (and
> - implement these planned facilities), it should act as if the response
> - carried a "Vary:{accept-headers}" header.  This means that the response
> - may be returned in reply to a subsequent request with Accept-* headers
> - identical to those in the current request.
> - 
> - 
> - 16.5.3 Variant-ID Use
> - If an origin server chooses to use the variant-ID mechanism, it assigns
> - a variant-ID (see section 7.12) to each distinct resource entity
> - (variant). This assignment can only be made by the origin server. It
> - then returns the appropriate variant-ID with each response that applies
> - to a specific resource entity (variant), using the ETag header (see
> - Error! Reference source not found.).
> - 
> - 
> - 
> - Fielding, Frystyk, Berners-Lee, Gettys, and Mogul  [Page 75]
> - 
> - 
> - 
> - 
> - INTERNET-DRAFT            HTTP/1.1      Friday, May 03, 1996
> - 
> - 
> - When sending an entity derived from a particular variant in a response,
> - an origin server SHOULD include a variant-ID identifying the variant in
> - the ETag header (see section Error! Reference source not found.).  This
> - variant-ID can be used for cache replacement and in conditional requests
> - on the generic resource. When a cache receives a successful response
> - with a variant-ID, it SHOULD use this information to replace any
> - existing cache entries for the same variant of the corresponding URI.
> - That is, it forms a cache key using the URI of the request and the
> - variant-ID of the response. If this key matches the key of an existing
> - cache entry, it SHOULD replace the existing entry with the new response
> - (subject to all of the other rules on caching). See section Error!
> - Reference source not found. for more details on update.
> - 
> - When a cache performs a conditional request on a generic resource, and
> - it has one or more cache entries for the resource that include variant-
> - IDs, the cache MUST transmit the (cache-validator, variant-ID) tuples in
> - the conditional request, using the variant-set mechanism (see section
> - 7.13). This tells the server which variants are currently in the
> - requester's cache.
> - 
> -   The client MAY choose to transmit only a subset of the (cache-
> -   validator, variant-ID) tuples corresponding to its cache entries
> -   for this resource.
> - 
> - When a server receives a conditional request that includes a variant-
> - set, and the server is able to reply with an appropriate variant
> - (either
> - because it is the origin server, or because it is an intermediate cache
> - that can properly implement the variant selection algorithm), once it
> - has selected the variant it should examine the elements of the supplied
> - variant-set. If one of these matches the variant-ID of the selected
> - variant, and if the cache validators match, the server SHOULD reply with
> - a 304 (Not Modified) response, including the variant-ID of the selected
> - variant. Otherwise, the server should reply as if the request were
> - unconditional.
> - 
> - The server may optionally use the variant-set information in its
> - selection algorithm. For example, if the selection algorithm yields
> - several variants with equal preference, and one of these is already in
> - the requester's cache, the server could select that variant and avoid an
> - extra data transfer. This is a performance optimization; otherwise, the
> - variant-selection mechanism is orthogonal to the variant-ID mechanism.
> - 
> - 
> --- 4700,4736 ----
>   missing), and must discard the other partial information.
>   
>   
> ! 16.5 Caching Negotiated Responses
>   
> ! Use of server-driven content negotiation (section 15), as indicated by
> ! the presence of a Vary header field in a response, alters the conditions
> ! and procedure by which a cache can use the response for subsequent requests.
>   
> ! A server MUST use the Vary header field (section 18.46) to inform a cache
> ! of what header field dimensions are used to select among multiple
> ! representations of a response. A cache can use the selected
> ! representation (the entity included with that particular response) for
> ! replying to subsequent requests on that resource only when the subsequent
> ! requests have the same or equivalent value for all header fields
> ! specified in the Vary response-header.  Requests with a different value
> ! for one or more of those header fields SHOULD be forwarded toward the
> ! origin server; if an entity tag was assigned to the representation,
> ! the forwarded request SHOULD be conditional and include the entity tag
> ! in an If-None-Match header field.
>   
> ! The Vary header field may also inform the cache that the representation
> ! was selected using criteria not limited to the request headers; in this
> ! case, a cache SHOULD NOT use the response in a reply to a subsequent
> ! request unless the cache relays the new request to the origin server in
> ! a conditional request and the server responds with 304 (Not Modified),
> ! including an entity tag or Content-Location that indicates which entity
> ! should be used.
>   
> + A cache MAY accumulate multiple representations of a given response.
> + If two such entities include the same Content-Location value, it is
> + recommended (not required) that the cache discard the older entity
> + (as determined by the Date header field of the two responses). 

REASON: All of the complexity associated with variant-id handling and
what caches must do within their storage management functions are no
longer needed.  Vary is fully-defined without any need to reference
Alternates, which means we don't need to define Alternates until it
is ready to be defined.  The above paragraphs match the intended
consensus of the conneg subgroup up to the last mail message on that
subgroup's hypermail archive.  The only difference is that I require
Vary to be sent even when Alternates is present, since that is the only
effective way to allow applications that only implement Vary to coexist
with applications that also implement Alternates.  The number of bytes
added to the average response is negligible.

> ***************
> *** 4844,4872 ****
>   If on a cache lookup there are two or more fresh entries that appear to
>   match the request, then the one with the most recent Date value MUST be
>   used.
> - 16.7.1 Plain Resources
> - If the cached response was for a plain resource (that is, the response
> - includes no Vary or Alternates headers), it matches if the Request-URI
> - of the request matches the Request-URI of the of the request that caused
> - the cached response to be stored. Request-URIs match if their canonical
> - forms (see section 7.2.3) are equal.
>   
> - 16.7.2 Generic Resources
> - If the cached response was for a generic resource (that is, the response
> - includes Vary, or Alternates headers), it matches if the Request-URI of
> - the request matches the Request-URI of the request that caused the
> - cached response to be stored, and the selecting request header field
> - values of the request match those of the request that caused the cached
> - response to be stored. (See section 18.46 on Vary, which defines the
> - canonical form for selecting request headers and the matching rules for
> - them.)
> - If the response contains "Vary: {other}", then the selecting request
> - header field values for its request are defined as never matching a set
> - of request headers.
> - 
>   16.8 Errors or Incomplete Response Cache Behavior
>   A cache that receives an incomplete response (for example, with fewer
> ! bytes of data than specified in a Content-length: header) may store the
>   response. However, the cache MUST treat this as a partial response.
>   Partial responses may be combined as described in section 16.4.4; the
>   result might be a full response or might still be partial. A cache MUST
> --- 4762,4771 ----
>   If on a cache lookup there are two or more fresh entries that appear to
>   match the request, then the one with the most recent Date value MUST be
>   used.
>   
>   16.8 Errors or Incomplete Response Cache Behavior
>   A cache that receives an incomplete response (for example, with fewer
> ! bytes of data than specified in a Content-length header) may store the
>   response. However, the cache MUST treat this as a partial response.
>   Partial responses may be combined as described in section 16.4.4; the
>   result might be a full response or might still be partial. A cache MUST

REASON: (see above)

> ***************
> *** 4993,5061 ****
>   updates and the problems arising from server, cache, or network failure
>   prior to write-back.
>   
> - 
> - 16.12  Generic Resources and HTTP/1.0 Proxy Caches
> - If the correct handling of responses from a generic resource (Section
> - 15) by HTTP/1.0 proxy caches in the response chain is important,
> - HTTP/1.1 origin servers can include the following Expires (Section
> - 18.22) response header in all responses from the generic resource:
> - 
> -      Expires: Thu, 01 Jan 1980 00:00:00 GMT
> - 
> - If this Expires header is included, the server should usually also
> - include a Cache-Control header for the benefit of HTTP/1.1 caches, for example
> - 
> -      Cache-Control: max-age=604800
> - 
> - which overrides the freshness lifetime of zero seconds specified by the
> - included Expires header.

REASON: That is completely bogus and unnecessary.  The protocol cannot
advocate defeat of its own features.  If it does, I will insist that
implementers ignore all such cache constraints, since they are clearly
contrary to the future of the Internet as a whole.  Furthermore, it
is incorrect to describe HTTP/1.0 as not including cache-control,
since any HTTP/1.0 application can (and should) implement it even if
they are not HTTP/1.1 compliant in other aspects.

If origin servers invent ugly hacks to handle older protocols, that is
their perogative.  There is no basis for making it a standard.

> - 
> - 16.13 Cache Replacement
> - If a new cacheable response (see sections 18.10.2, 16.2.6, 16.2.8 and
> - 16.8) is received from a plain resource while any existing responses for
> - the same resource are cached, the cache MUST NOT return any of those
> - older responses to any  future requests for the resource.
> - 
> -   Note: a new response that has an older Date header value than
> -   existing cached responses is not cacheable.
> - 
> - If a new cacheable response is received from a generic resource with a
> - certain variant-ID while any old responses with the same variant-ID for
> - the same resource are cached, the cache MUST NOT return any of those old
> - responses to any future requests for the resource.
> - 
> -   Note: In some cases, this may mean that the cache chooses to delete
> -   the old response(s) from cache storage to recover space. However,
> -   note that there will never be a new response to signal that a
> -   variant-ID is no longer in use. It is expected that the cache's
> -   update heuristics will eventually cause such old responses to be
> -   deleted.
> - 
> - The cache SHOULD use the new response to reply to the current request.
> - It may insert it into cache storage and may, if it meets all other
> - requirements, use it to respond to any future requests that would
> - previously have caused the old response to be returned. If it inserts
> - the new response into cache storage it should follow the rules in
> - section 16.4.3.
> - 

REASON: HTTP does not place requirements on how a cache manages its own
storage process.

> - 
> - 16.14 Caching of Negative Responses
> - Caching of negative responses has often been a significant performance
> - advantage in distributed systems.  In some future draft or specification
> - we may have more to say about negative caching.
> - 

REASON: Completely bogus -- caching of any response is already defined
by this specification.

> --- 4892,4897 ----
> ***************
> *** 5606,5640 ****
>   
> - 18.8 Alternates
> - The Alternates response-header field is used by origin servers to signal
> - that the resource identified by the current request has the capability
> - to send different responses depending on the accept headers in the
> - request message.  This has an important effect on cache management,
> - particularly for caching proxies which service a diverse set of user
> - agents.  This effect is covered in section 18.46.
> - 
> -        Alternates           = "Alternates" ":" opaque-field
> - 
> -        opaque-field         = field-value
> - 
> - The Alternates header is included into HTTP/1.1 to make HTTP/1.1 caches
> - compatible with a planned content negotiation mechanism.  HTTP/1.1
> - allows a future content negotiation standard to define the format of the
> - Alternates header field-value, as long as the defined format satisfies
> - the general rules in section 18.8.
> - 
> - To ensure compatibility with future experimental or standardized
> - software, caching HTTP/1.1 clients MUST treat all Alternates headers in
> - a response as synonymous to the following Vary header:
> - 
> -          Vary: {accept-headers}
> - 
> - and follow the caching rules associated with the presence of this Vary
> - header, as covered in Section 18.46.  HTTP/1.1 allows origin servers to
> - send Alternates headers under experimental conditions.
> - 
> - 
>   18.9 Authorization
> --- 5442,5447 ----

REASON: Moved to appendix -- no longer overlaps with Vary functionality.

> ***************
> *** 6152,6159 ****
>   
>   Language tags are defined in section 7.10. The primary purpose of
>   Content-Language is to allow a selective consumer to identify and
> ! differentiate resource variants according to the consumer's own
> ! preferred language. Thus, if the body content is intended only for a
>   Danish-literate audience, the appropriate field is
>   
>          Content-Language: dk
> --- 5959,5966 ----
>   
>   Language tags are defined in section 7.10. The primary purpose of
>   Content-Language is to allow a selective consumer to identify and
> ! differentiate entities according to the consumer's own
> ! preferred language. Thus, if the content is intended for a
>   Danish-literate audience, the appropriate field is
>   
>          Content-Language: dk

REASON: terminology

> ***************
> *** 6184,6190 ****
>   
> ! Content-Language MAY be applied to any media type -- it SHOULD not be
>   limited to textual documents.
>   
> --- 5991,5997 ----
>   
> ! Content-Language MAY be applied to any media type -- it is not 
>   limited to textual documents.

REASON: spec-ese

> ***************
> *** 6475,6506 ****
>   
>   
>   
> ! 18.21 ETag
> ! The ETag header is used to transmit entity tags with variant id's in
> ! HTTP/1.1 responses.
>   
> -       ETag = "ETag" ":" etag-info
> -       etag-info = entity-tag [ ";" variant-id ]
> - 
>   Examples:
>   
>         ETag: "xyzzy"
> !       ETag: "xyzzy"/W
> !       ETag: "xyzzy";"3"
> !       ETag: "xyzzy"/W;"3"
> !       ETag: ""
>   
> -   Note that the variant-id is not part of the entity tag. The ETag
> -   field is used to transmit a variant-id simply as a matter of
> -   compact representation of responses.
>   
> --- 6282,6300 ----
>   
>   
>   
> ! 18.21 Etag
>   
> ! The Etag entity-header field defines the entity tag for the associated
> ! entity.  The entity tag may then be used for comparison with other
> ! entities from the same resource.
>   
> +       ETag = "ETag" ":" entity-tag
>   
>   Examples:
>   
>         ETag: "xyzzy"
> !       ETag: W/"xyzzy"
>   

REASON: (see above)

> ***************
> *** 6690,6814 ****
>   
>   
>   18.26 If-Match
>   The If-Match request-header field is used with a method to make it
> ! conditional. A client that has a cache entry for the relevant entity
> ! supplies the associated entity tag using the If-Match header; if this
> ! entity tag matches the server's current entity tag for the entity, the
> ! server SHOULD perform the requested operation as if the If-Match header
> ! were not present.
>   
> ! If the entity tags do not match, the server MUST NOT perform the
> ! requested operation, and MUST return a 412 (Precondition failed)
> ! response with no Entity-Body. This behavior is most useful when the
> ! client wants to prevent an updating method, such as PUT or POST, from
> ! modifying a resource entity  that  has changed since the client last
> ! checked it.
>   
> ! When the If-Match header is used, the server should use the strong
> ! comparison function (see section 18.26) to compare entity tags.
>   
> ! If the If-Match header is used to make a conditional request on generic
> ! resource, it may be used to pass a set of validators.  This is done
> ! using the variant-set mechanism if the client has variant IDs for the
> ! corresponding cache entries (see sections 16.5.3 and 7.13 ).  The server
> ! selects the appropriate variant based on other request headers; if the
> ! variant-ID for that resource entity is listed in the If-Match header,
> ! and if the entity-tag associated with that variant-ID in the header
> ! matches the current entity-tag of the resource entity, then the
> ! requested operation SHOULD be performed.  Otherwise, it MUST NOT be
> ! performed.
>   
> !       If-Match = "If-Match" ":" if-match-rhs
> !       if-match-rhs = opaque-validator | variant-set
>   
> ! An updating request (e.g., a PUT or POST) on a generic  resource should
> ! include only one variant-set-item, the one associated with the
> ! particular variant whose value is being conditionally updated.
>   
> ! Examples of plain resource  form:
>   
>          If-Match: "xyzzy"
> !        If-Match: "xyzzy"/W
>   
> - Examples of generic resource  form:
> - 
> -        If-Match: "xyzzy";"4"
> -        If-Match: "xyzzy";"3", "r2d2xxxx";"5", "c3piozzzz";"7"
> -        If-Match: "xyzzy"/W; "3", "r2d2xxxx"/W; "5", "c3piozzzz"/W; "7"
> - 
> - If the request would, without the If-Match header, result in anything
> - other than a 2xx status, then the If-Match header is ignored.
> - 
>   The purpose of this feature is to allow efficient updates of cached
>   information with a minimum amount of transaction overhead. It is also
>   used, on updating requests, to prevent inadvertent modification of the
> ! wrong variant of a resource.
>   
>   
> ! 18.27 If-NoneMatch
> ! The If-NoneMatch request-header field is used with a method to make it
> ! conditional. A client that has a cache entry for the relevant entity
> ! supplies the associated entity tag using the If-NoneMatch header; if
> ! this entity tag matches the server's current entity tag for the entity,
> ! the server SHOULD return a 304 (Not Modified) response without any
> ! Entity-Body.
>   
> ! If the entity tags do not match, the server should treat the request as
> ! if the If-NoneMatch header was not present.
>   
> ! See section 18.26 for rules on how to determine if two entity tags
> ! match.
>   
> ! If the If-NoneMatch header is used to make a conditional request on
> ! generic resource, it may be used to pass a set of validators.  This is
> ! done using the variant-set mechanism if the client has variant IDs for
> ! the corresponding cache entries (see sections 16.5.3 and 7.13).  The
> ! server selects the appropriate variant based on other request headers;
> ! if the variant-ID for that resource entity is listed in the
> ! If-NoneMatch
> ! header, and if the entity-tag associated with that variant-ID in the
> ! header matches the current entity-tag of the resource entity, then the
> ! requested operation SHOULD NOT be performed.  Otherwise, it SHOULD be
> ! performed.
>   
> !       If-NoneMatch = "If-NoneMatch" ":" if-nonematch-rhs
> !       if-nonematch-rhs = opaque-validator | variant-set
>   
> ! Examples of plain resource form:
>   
> !        If-NoneMatch: "xyzzy"
> !        If-NoneMatch: "xyzzy"/W
>   
> ! Examples of generic resource form:
>   
> !        If-NoneMatch: "xyzzy";"4"
> !        If-NoneMatch: "xyzzy";"3", "r2d2xxxx";"5", "c3piozzzz";"7"
> !        If-NoneMatch: "xyzzy"/W; "3", "r2d2xxxx"/W; "5", "c3piozzzz"/W;7
>   
> - If the request would, without the If-NoneMatch header, result in
> - anything other than a 2xx status, then the If-NoneMatch header is
> - ignored.
> - 
>   The purpose of this feature is to allow efficient updates of cached
> ! information with a minimum amount of transaction overhead.
>   
>   
>   18.28 If-Range
> --- 6484,6574 ----
>   
>   
>   18.26 If-Match
> + 
>   The If-Match request-header field is used with a method to make it
> ! conditional. A client that has one or more entities previously obtained
> ! from the resource can verify that one of those entities is current
> ! by including a list of their associated entity tags in the If-Match
> ! header field.  As a special case, the value "*" matches any current
> ! entity of the resource.
>   
> !          If-Match = "If-Match" ":" ( "*" | 1#entity-tag )
>   
> ! If any of the entity tags match the entity tag of the entity that
> ! would have been returned in the response to a similar GET request
> ! on that resource, or if "*" is given and any current entity exists
> ! for that resource, then the server MAY perform the requested method.
> ! as if the If-Match header field did not exist.
>   
> ! A server MUST use the strong comparison function (see section 18.26)
> ! to compare the entity tags in If-Match.
>   
> ! If none of the entity tags match, or if "*" is given and no current
> ! entity exists, the server MUST NOT perform the requested method, and
> ! MUST return a 412 (precondition failed) response. This behavior is
> ! most useful when the client wants to prevent an updating method,
> ! such as PUT, from modifying a resource that has changed since the
> ! client last retrieved it.
>   
> ! If the request would, without the If-Match header field, result in
> ! anything other than a 2xx status, then the If-Match header MUST be
> ! ignored.
>   
> ! Examples include:
>   
>           If-Match: "xyzzy"
> !         If-Match: W/"xyzzy"
> !         If-Match: xyzzy", "r2d2xxxx", "c3piozzzz"
>   
>   The purpose of this feature is to allow efficient updates of cached
>   information with a minimum amount of transaction overhead. It is also
>   used, on updating requests, to prevent inadvertent modification of the
> ! wrong version of a resource.
>   
>   
> ! 18.27 If-None-Match
>   
> ! The If-None-Match request-header field is used with a method to make it
> ! conditional. A client that has one or more entities previously obtained
> ! from the resource can verify that none of those entities is current
> ! by including a list of their associated entity tags in the If-None-Match
> ! header field.  As a special case, the value "*" matches any current
> ! entity of the resource.
>   
> !         If-None-Match = "If-None-Match" ":" ( "*" | 1#entity-tag )
>   
> ! If any of the entity tags match the entity tag of the entity that
> ! would have been returned in the response to a similar GET request
> ! on that resource, or if "*" is given and any current entity exists
> ! for that resource, then the server MUST NOT perform the requested method.
> ! Instead, if the request method was GET or HEAD, the server SHOULD
> ! respond with a 304 (not modified) response, including the cache-related
> ! entity-header fields (particularly Etag) of one of the entities that
> ! matched.  For all other request methods, the server MUST respond with
> ! a status of 412 (precondition failed).
>   
> ! See section 18.26 for rules on how to determine if two entity tags
> ! match.  The weak comparison function can only be used with GET or
> ! HEAD requests.
>   
> ! If none of the entity tags match, or if "*" is given and no current
> ! entity exists, then the server MAY perform the requested method as if
> ! the If-None-Match header field did not exist.
>   
> ! If the request would, without the If-None-Match header field, result
> ! in anything other than a 2xx status, then the If-None-Match header
> ! MUST be ignored.
>   
> ! Examples:
>   
> !         If-None-Match: "xyzzy"
> !         If-None-Match: W/"xyzzy"
> !         If-None-Match: "xyzzy", "r2d2xxxx", "c3piozzzz"
>   
>   The purpose of this feature is to allow efficient updates of cached
> ! information with a minimum amount of transaction overhead. It is also
> ! used, on updating requests, to prevent inadvertent modification of a
> ! resource which was not known to exist.

REASON: I rewrote these sections to simplify the verbage, remove redundancy,
and replace the "" entity tag hack with a clearly defined "*".

> ***************
> *** 6824,6830 ****
>   me the part(s) that I am missing; otherwise, send me the entire new
>   entity.'"
>   
> !         Range-If = "Range-If" ":" (if-valid-rhs | HTTP-date)
>   
>   If the client has no entity tag for a plain resource, but does have a
>   Last-Modified date, it may use that date in a If-Range header.  (The
> --- 6584,6590 ----
>   me the part(s) that I am missing; otherwise, send me the entire new
>   entity.'"
>   
> !         If-Range = "If-Range" ":" ( entity-tag | HTTP-date)
>   
>   If the client has no entity tag for a plain resource, but does have a
>   Last-Modified date, it may use that date in a If-Range header.  (The

REASON: The old BNF was wrong.

> ***************
> *** 6843,6857 ****
>   
>   
>   18.29 If-Unmodified-Since
>   The If-Unmodified-Since request-header field is used with a method to
> ! make it conditional. If the requested resource entity has not been
> ! modified since the time specified in this field, the server should
>   perform the requested operation as if the If-Unmodified-Since header
>   were not present.
>   
>   If the requested resource entity has been modified since the specified
>   time, the server MUST NOT perform the requested operation, and MUST
> ! return a 412 (Precondition Failed) response with no Entity-Body.
>   
>         If-Unmodified-Since = "If-Unmodified-Since" ":" HTTP-date
>   
> --- 6603,6618 ----
>   
>   
>   18.29 If-Unmodified-Since
> + 
>   The If-Unmodified-Since request-header field is used with a method to
> ! make it conditional. If the requested resource has not been modified
> ! since the time specified in this field, the server should
>   perform the requested operation as if the If-Unmodified-Since header
>   were not present.
>   
>   If the requested resource entity has been modified since the specified
>   time, the server MUST NOT perform the requested operation, and MUST
> ! return a 412 (Precondition Failed) response.
>   
>         If-Unmodified-Since = "If-Unmodified-Since" ":" HTTP-date
>   

REASON: IUS applies to the resource, and 412 does contain an entity.

> ***************
> *** 7106,7112 ****
>        words, the response carries a status code of 206 (Partial Content)
>        instead of 200 (OK).
>     .  The presence of a Range header in a conditional GET (a request
> !      using one or both of If-Modified-Since and If-NoneMatch, or one or
>        both of If-Unmodified-Since and If-Match) modifies what is returned
>        if the GET is otherwise successful and the condition is true.  It
>        does not affect the 304 (Not Modified) response returned if the
> --- 6867,6873 ----
>        words, the response carries a status code of 206 (Partial Content)
>        instead of 200 (OK).
>     .  The presence of a Range header in a conditional GET (a request
> !      using one or both of If-Modified-Since and If-None-Match, or one or
>        both of If-Unmodified-Since and If-Match) modifies what is returned
>        if the GET is otherwise successful and the condition is true.  It
>        does not affect the 304 (Not Modified) response returned if the

REASON: Typo.

> ***************
> *** 7321,7490 ****
>   
>   
>   18.46 Vary
> - The Vary response-header field is used by an origin server to signal
> - that the resource identified by the current request is a generic)
> - resource.  A generic resource has multiple entities associated with it,
> - all of which are representations of the content of the resource.  If a
> - GET or HEAD request on a generic resource is received, the origin server
> - will select one of the associated entities as the entity best matching
> - the request.  Selection of this entity is based on the contents of
> - particular header fields in the request message, or on other information
> - pertaining to the request, like the network address of the sending
> - client.

Let's count the ways this is wrong:
1. "generic resource" is a poor term to refer to a response that varies.
2. whether or not the resource has multiple entities is irrelevant -- what
   is important is that the response entity has multiple, equivalent
   representations.
3. content negotiation applies to more than just "GET or HEAD".
4. the last sentence is grammatically goofy.

> ! A resource being generic has an important effect on cache management,
> ! particularly for caching proxies which service a diverse set of user
> ! agents.  All 200 (OK) responses from generic resources MUST contain at
> ! least one Vary header (section 18.46) or Alternates header (section
> ! 18.8) to signal variance.

5. content negotiation applies to ANY response, regardless of status.

> ! If no Vary headers and no Alternates headers are present in a 200 (OK)
> ! response, then caches may assume, as long as the response is fresh, that
> ! the resource in question is plain, and has only one associated entity.
> ! Note however that this entity can still change through time, as possibly
> ! indicated by a Cache-Control response header (section 18.10).

6. Alternates is orthogonal to Vary.
7. The last sentence doesn't say anything.

> ! After selection of the entity best matching the current request, the
> ! origin server will usually generate a 200 (OK) response, but it can also
> ! generate other responses like 206 (Partial Content) or 304 (Not
> ! Modified) if headers which modify the semantics of the request, like
> ! Range (section 18.38) or If-Match (section 18.26), are present.  An
> ! origin server need not be capable of selecting an entity for every
> ! possible incoming request on a generic resource; it can choose to
> ! generate a 3xx (redirection) or 4xx (client error) type response for
> ! some requests.

8. A totally ineffective description of what isn't required by HTTP anyway.

> ! In a request message on a generic resource, the selecting request
> ! headers are those request headers whose contents were used by the origin
> ! server to select the entity best matching the request. The Vary header
> ! field specifies the selecting request headers and any other selection
> ! parameters that were used by the origin server.

9. FATAL description of Vary in contradiction of all past discussion of
the Vary header field, including David Robinson's draft at
<http://www.ast.cam.ac.uk/~drtr/vary.html>, and directly contradicting
the previously cited consensus on conneg.  Vary specifies the list of
header fields over which the representation may vary on any future request.
Vary would serve no useful purpose if that were not true.

> !        Vary                 = "Vary" ":" 1#selection-parameter
>   
> -        selection-parameter  = request-header-name
> -                             | "{accept-headers}"
> -                             | "{other}"
> -                             | "{" extension-parameter "}"
> - 
> -        request-header-name  = field-name
> - 
> -        extension-parameter  = token
> - 
> - The presence of a request-header-name signals that the request-header
> - field with this name is selecting.  Note that the name need not belong
> - to a request-header field defined in this specification, and that header
> - names are case-insensitive.  The presence of the "{accept-headers}"
> - parameter signals that all request headers whose names start with
> - "accept" are selecting.

This is not a useful short-hand.  Please note that almost all negotiated
resources vary on one or two parameters.  Furthermore, this use of the
bag syntax {curly braces} is contrary to the overall design of having that
syntax represent hierarchy.

> - The inclusion of the "{other}" parameter in a Vary field signals that
> - parameters other than the contents of request headers, for example the
> - network address of the sending party, play a role in the selection of
> - the response.

It is cheaper, easier, and more accurate to use "*" for this.

> -   Note: This specification allows the origin server to express that
> -   other parameters were used, but does not allow the origin server to
> -   specify the exact nature of these parameters.  This is left to
> -   future extensions.
> - 
> - If an extension-parameter unknown to the cache is present in a Vary
> - header, the cache MUST treat it as the "{other}" parameter. ...

This "extensibility mechanism" is therefore not extensible.  The only
thing it accomplishes is added complexity.  We gain the exact same
extensibility via a separate field, e.g.

    Vary: "*"
    My-extension: fred

where My-extension is defined as modifying the semantics of Vary.
In other words, the extension-parameter is completely unnecessary.
By removing it, we can remove the hokey use of braces and any trace
of confusion over how to parse Vary.

> -                                                              If multiple
> - Vary and Alternates header fields are present in a response, these MUST
> - be combined to give all selecting parameters.

That is true of all header fields and does not need to be said.

> - The field name "Host" MUST never be included in a Vary header; clients
> - MUST ignore it if it is present.  The names of fields which change the
> - semantics of a GET request, like "Range" and "If-Match" MUST also never
> - be included, and MUST be ignored when present.

If they aren't useful, they won't be used.  This is just overspecification.

> - Servers which use access authentication are not obliged to send "Vary:
> - Authorization" headers in responses.  It MUST be assumed that requests
> - on authenticated resources can always produce different responses for
> - different users.  Note that servers can signal the absence of
> - authentication by including "Cache-Control: public" header in the
> - response.

The first statement is wrong, the second unnecessary.  AA does not imply
that the content is specific to a particular user (which is what
Vary: Authentication says).  AA only indicates that permission is required.

> - A cache MAY store and refresh 200 (OK) responses from a generic resource
> - according to the rules in section 16.4.  The partial entities in 206
> - (Partial Content) responses from generic resources MAY also be used by
> - the cache.

Irrelevant redundancy.

> - When getting a request on a generic resource, a cache can only return a
> - cached 200 (OK) response to one of its clients in two particular cases.
> - 
> - First, if a cache gets a request on a generic resource for which it has
> - cached one or more responses with Vary or Alternates headers, it can
> - relay that request towards the origin server, adding an If-NoneMatch
> - header listing the etag-info values in the ETag headers (section Error!
> - Reference source not found.) of the cached responses which have
> - variant-
> - IDs.  If it then gets back a 304 (Not Modified) response with the etag-
> - info of a cached 200 (OK) response in its ETag header, it can return
> - this cached 200 (OK) response to its client, after merging in any of the
> - 304 response headers as specified in section 16.4.2.
> - 
> - Second, if a cache gets a request on a generic resource, it can return
> - to its client a cached, fresh 200 (OK) response which has Vary or
> - Alternates headers, provided that
> - 
> - 
> -   .  the Vary and Alternates headers of this fresh response specify that
> -      only request header fields are selecting parameters,
> - 
> -   .  the specified selecting request header fields of the current
> -      request match the specified selecting request header fields of a
> -      previous request on the resource relayed towards the origin
> - server,
> - 
> -   .  this previous request got a 200 (OK) or 304 (Not Modified) response
> -      which had the same etag-info value in its ETag header as the
> -      cached, fresh 200 (OK) response.

Relevant, but still redundant.  This is wrong in ignoring error responses,
and is already specified in the caching section anyway.

> - Two sequences of selecting request header fields match if and only if
> - the first sequence can be transformed into the second sequence by only
> - adding or removing whitespace at places in fields where this is allowed
> - according to the syntax rules in this specification.

Does not belong here.  If it is indeed necessary, it should be in the
section on message header fields.

> - If a cached 200 (OK) response MAY be returned to a request on a generic
> - resource which includes a Range request header, then a cache MAY also
> - use this 200 (OK) response to construct and return a 206 (Partial
> - Content) response with the requested range.
> - 
> -   Note: Implementation of support for the second case above is mainly
> -   interesting in user agent caches, as a user agent cache will
> -   generally have an easy way of determining whether the sequence of
> -   request header fields of the current request equals the sequence
> -   sent in an earlier request on the same resource.  Proxy caches
> -   supporting the second case would have to record diverse sequences
> -   of request header fields previously relayed; the implementation
> -   effort associated with this may not be balanced by a sufficient
> -   payoff in traffic savings.  A planned specification of a content
> -   negotiation mechanism will define additional cases in which proxy
> -   caches can return a cached 200 (OK) response without contacting the
> -   origin server.  The implementation effort associated with support
> -   for these additional cases is expected to have a much better
> -   cost/benefit ratio.
> - 
Irrelevant (to Vary) and redundant.

> --- 7082,7130 ----
>   
>   
>   18.46 Vary
>   
> ! The Vary response-header field is used by a server to signal that the
> ! response entity was selected from the available representations of the
> ! response using server-driven negotiation (section 15).  The Vary field
> ! value indicates either that the given set of header fields encompass
> ! the dimensions over which the representation might vary, or that the
> ! dimensions of variance are unspecified ("*") and thus may vary over
> ! any aspect of future requests.
>   
> !        Vary  = "Vary" ":" ( "*" | 1#field-name )
>   
> ! An HTTP/1.1 server MUST include an appropriate Vary header field with
> ! any response that is subject to server-driven negotiation.  Doing so
> ! allows a cache to properly interpret future requests on that resource
> ! and informs the user agent about the presence of negotiation on that
> ! resource.
>   
> ! A Vary field value of "*" signals that parameters other than the
> ! contents of request-header fields (e.g., the network address of the
> ! client) play a role in the selection of the response representation.
> ! Subsequent requests on that resource can only be properly interpreted
> ! by the origin server, and thus a cache SHOULD forward a (possibly
> ! conditional) request even when it has a fresh response cached from
> ! a prior request on the resource.
>   
> ! A Vary field value consisting of a list of field-names signals that
> ! the representation selected for the response is based on a selection
> ! algorithm which considers ONLY the listed request-header field values in
> ! selecting the most appropriate representation.  This selection algorithm
> ! MAY be assumed to remain unchanged (and thus apply to future requests)
> ! for the duration of time in which the response is fresh.
>   
> + The field-names given are not limited to the set of standard
> + request-header fields defined by this specification. Field names are
> + case-insensitive.
>   
> + The value of the Vary field itself MUST NOT vary on any aspect of the
> + request other than the requested resource (Request-URI and Host).
> + In other words, two simultaneous requests on the same resource must 
> + result in identical Vary fields (if any), regardless of the header
> + fields in those requests, if the response status codes are identical.

I hope people note how much simpler, shorter, and more understandable
this description is than what was in draft 03.  Moreover, it doesn't
prevent error responses from being negotiated on, e.g., language.

> ***************
> *** 7890,7896 ****
>   An approach that limits the loss of privacy would be for a user agent to
>   omit the sending of  Accept-Language headers by default, and to ask the
>   user whether it should start sending Accept-Language headers to a server
> ! if it detects, by looking for any Vary or Alternates response headers
>   generated by the server, that such sending could improve the quality of
>   service.
>   
> --- 7530,7536 ----
>   An approach that limits the loss of privacy would be for a user agent to
>   omit the sending of  Accept-Language headers by default, and to ask the
>   user whether it should start sending Accept-Language headers to a server
> ! if it detects, by looking for any Vary response header field(s)
>   generated by the server, that such sending could improve the quality of
>   service.
>   
> ***************
> *** 8629,8634 ****
> --- 8269,8293 ----
>   
>   23.5.2 Additional Header Field Definitions
>   
> + 23.5.2.1 Alternates
> + 
> + The Alternates response-header field has been proposed as a means for
> + the origin server to inform the client about other available 
> + representations of the requested resource, along with their
> + distinguishing attributes, and thus providing a more reliable means for
> + a user agent to perform subsequent selection of another representation
> + which better fits the desires of its user (described as agent-driven
> + negotiation in section 15.2).
> + 
> + The Alternates header field is orthogonal to the Vary header field in
> + that both may coexist in a message without affecting the interpretation
> + of the response or the available representations.  It is expected that
> + Alternates will provide a significant improvement over the server-driven
> + negotiation provided by the Vary field for those resources that vary
> + over common dimensions like type and language.
> + 
> + The Alternates header field will be defined in a future specification.
> + 
>   23.5.2.1 Content-Version
>   
>   
> ***************
> *** 8737,8746 ****
>   
>   
>   23.5.2.4 URI
>   The URI header field has, in past versions of this specification, been
>   used as a combination of the existing Location, Content-Location, and
> ! Alternates header fields. Its primary purpose has been to include a list
> ! of additional URIs for the resource, including names and mirror
>   locations.  However, it has become clear that the combination of many
>   different functions within this single field has been a barrier to
>   consistently and correctly implementing any of those functions.
> --- 8396,8407 ----
>   
>   
>   23.5.2.4 URI
> + 
>   The URI header field has, in past versions of this specification, been
>   used as a combination of the existing Location, Content-Location, and
> ! Vary header fields, as well as the future Alternates field (above).
> ! Its primary purpose has been to include a list of additional URIs for
> ! the resource, including names and mirror
>   locations.  However, it has become clear that the combination of many
>   different functions within this single field has been a barrier to
>   consistently and correctly implementing any of those functions.
> ===========
> end of diff

I would not have proposed these changes at this late a date if I did
not think that they were absolutely necessary.  I did not make them
directly to the document, nor did I ask Jim to change the draft before
this was reviewed by the WG, as might be implied by Koen's messages.
The problems outlined above are major errors.  For example, they would
have made it impossible for the Apache server to make use of HTTP/1.1.
I believe the same would be true of Spyglass's server.

HTTP/1.1 will not go forward until these problems are fixed.  I have
provided a complete set of changes to fix those problems.  It is now
up to the WG to decide if my suggested changes should be made to the
next draft, or if some other set of changes should be made.


 ...Roy T. Fielding
    Department of Information & Computer Science    (fielding@ics.uci.edu)
    University of California, Irvine, CA 92717-3425    fax:+1(714)824-4056
    http://www.ics.uci.edu/~fielding/

Received on Tuesday, 28 May 1996 02:28:34 UTC