Vary VS URI with an eye towards caching

The following verbiage is my idea of what the content negotiation scheme now
looks like, based on 1) my opinions, 2) my opinions of the HTTP 1.1 draft,
3) my opinions of Koen's posts, 4) my opinions of the recent content
negotiation meeting.

The short version: We want to add a Vary: header.

The long version:  (Anyone who reads this deserves a virtual cookie.  God
this is long.)

1) Background
The latest draft of HTTP 1.1 added a new improved URI header that allowed
server to list the variants of a particular resource it had available on
it's system.  This new URI header also included some meta-information about
each variant: just enough for those who see the response to duplicate the
content negotiation process at a later date, and come up with the same
result the server would on any request.  Let's call this "transparent
negotiation" because all the information needed by the proxy to negotiate is
transparent/known, both data and algorithm.

2) Benefit
By giving the power to a proxy to do it's own content negotiation, the proxy
can avoid contacting the server on every request to have that server do the
content negotiation for the request.  The proxy has all the information it
needs: the variant picking algorithm (from the spec), the Accept: headers to
represent the UAs desires, and the server's list of variants w/
meta-information and qs values.  It can compute variants from cache and
serve them up (or send the request through if it doesn't have the variant,
and then the proxy can add the  variant to it's servable/cachable pool).

3) Problem
Larry Manister pointed out that many servers would not want to list all
possible variants.  A WWW image server might be able to convert to 200
different file formats on the fly.  In this case the URI: picking scheme
wouldn't work for proxies ability because of lack of data.  Other WWW
servers might want to send different variants based on User Agent: headers,
or Cookie: headers, or whatever.  In this case the URI: picking scheme
wouldn't work for proxies ability because of lack of the right algorithm.  A
proxy has to know about either one of these cases to be aware that it must
contact the server on every subsequent request (to let that server do the
negotiation).

4) Soution
Add back in the Vary: header.  This is the server's way of saying to the
proxy either 1) "I'm not telling you all the data, it's too much of a burden
to me" or "I have my own variant-picking algorithm."  When the server sends
a Vary: it is also effectively saying "You must contact me on each request
to let me choose the variant."  Let's call negotiating via a Vary: scheme
"opaque negotiation" because the proxy doesn't know everything.  Contacting
the origin server when faced with lack of data or algorithm is unavoidable,
but what would be worse would be if the proxy could not even cache responses
because it had no way of verifying it's cached item was the right variant
response.

4) Solution for caching transparent negotiation, AKA the URI scheme:
I've addressed how I think proxy caches for the URI scheme should behave in
a previously sent, but not widely circulated message:

>For uncached resources:
>I expect the caching proxy to make the request to the next server (origin 
>server or proxy) then store URI headers on the response, associate them with
>the requested URL, cache that requested URL's response, and simultaneously
>cache that response under the URL noted in the Location: response header. To
>restate: it caches the response both under the original "variable" URL (with
>extra URI info) and under the Location: URL.
>
>For cached resources:
>Then, on subsequent requests for:
>A) the URL with the associated URIs - I expect the proxy to apply the Chapter
>12 algorithm based on the Accept-* request headers it receives.  Once it has
>computed the correct variant, it should 1) search it's cache for that variant
>and send the response, or 2) go to the next server if it does not have that
>variant.
>B) the Location: headered URL - serve it. 

5) Other proposals:
Koen Holtman introduced a "Send-No-Body-For:" header for the purpose of
negotiation on URI headers, but he was under the influence of User-Agent
negotiation which I contend can't be easily done under the
transparent-proxy-has-all-data-and- algorithms-URI scheme.  It should be
done under the Vary: scheme.  As he intended it to work under some URI
scheme, he sought "Send-No-Body-For: URL".  Under my thoughts on transparent
negotiation, one would never make a request with "Send-No-Body-For: URL".
It would be redundant, if the proxy had that variant, it would be serving
it.  Under transparent negotiation, the proxy knows that variant is
servable, it doesn't need approval from the origin server.

6) Caching background:
Jeffrey said this in the caching group, and I thought the conneg people
should be exposed to it:
>We may find it impossible to allow the proxy to pick the right
>variant from its cache in every case, and unacceptable to require
>it to contact the server in every case.  But this may simply mean
>that we need to split the problem into "easy" cases and "hard" cases
>(or perhaps "easy", "hard", and "impossible" cases) and define
>somewhat different proxy algorithms for each class of cases.
>
>For example, an easy case might be "the server says there is exactly
>one variant, so the proxy always returns it".  A hard case might
>be "the server provides enough information to choose between
>variants, and so the proxy makes the choice."  An impossible case
>might be "the proxy is unable to make the choice, and so must
>present the request to the server, perhaps along with some
>indication of what possible responses it already has in its cache."

The last line is key.  It mimics what Koen was trying to do with
Send-No-Body-For.  I see Vary: as the method for the 'impossible' case, and
URI: as the method for the 'hard' case.

---
OK, everything from here is flakey and needs to be worked out:
---

7) Problem
Larry doesn't think a "Send-No-Body-For" construct should have URLs as
parameters; it should have some other unique content-id value.  This concern
stems from the idea that not all variants are nameable.  Separately, I have
come to the conclusion that if all variants are nameable, and the algorithm
is exposed properly, "Send-No-Body-For: URL" is unnecessary. (See #5).

8) Solution - Maybe?
The Cache-Validator: header as I understand it could be sent on subsequent
requests for an opaquely negotiated resource.  Now instead of asking "Is my
copy of the resource still good?" it is effectively asking "Is my copy of
the variant of the resource still good AND the variant you would give me?"
Besides possibly overloading the Cache-Validator header, this yields a
number of proxy implementation issues.  A proxy would have to decide if it
wants to save multiple variants under the same URI, each with different
opaque variant IDs.  If it did that, we'd have to allow multiple
Cache-Validator: headers.  (Which mean comma delimited when they get
squished for CGI environment variables, which means Cache-Validator values
must be defined to not allow commas.  Doh!)

 a) Now a server has to send back a Cache-Validator header right?  Hopefully one
      that matches the request, and hence comes back with a Not Modified or some
      other 20X You're Cool Dude no-entity response?
 b) Does the proxy still store the other non-matched variants?
 c1) Do we need to give the server a way of saying "clear all your variants"?
 c2) Do we need to give the serve a way of saying "clear these particular
      variants"?  (Presumably they'll clear eventually anyway as the proxy runs
      out of space and/or pushes them out of their working set.)
 d) Do cache-control messages (or ANY headers on the response for that
matter) on
      the opaquely negotiated response apply to all variants, just the
      server-chosen variant, or just the base resource 'negotiation port'?
 e) What about renegade proxies who serve variants willy-nilly without
contacting
      the server?  etc...
 f) Are we getting too complicated to hope for consistant implementations?
-----
Dan DuBois, Software Animal             http://www.spyglass.com/~ddubois/
		I absolutely do not speak for Spyglass.

Received on Saturday, 6 January 1996 02:57:30 UTC