Re: Content negotiation and Vary (1.3, 12, 13.5, 14.43)

>>[is Koen]
>>In the new pre-04 draft, the term `content negotiation' not only used
>>for a) the selection of the representation _of_ a resource to be
>>included in a successful response, as it was in the 00, 01, 02 and 03
>>drafts, but also for b) the selection of the representation of an
>>(error) message _about_ a resource.

An error message is a representation of a resource (not its good side,
but still a representation).  Drafts 00 and 01 did not exclude error
messages from being negotiated on purpose.  Draft 04a is just making
that explicit.

>[is jg]
> HTTP transfers entities; some come from resources, some contain
> information from servers the state of requests (often errors,
> or other useful information for end users.  It makes no
> sense in terms of the protocol to have to treat entities
> differently from each other.

Exactly.  All resources map a request (consisting of method, identifier,
request-header fields, and sometimes an entity) to a response
(consisting of a status code, response-header fields, and sometimes
an entity). 

Server-driven negotiation occurs when the server varies the response
entity for a particular method*identifier*request-entity*status-code
combination according to the value of the request-header fields,
or something external to the normal request parameters above (the "*" case).
The client needs to be notified that this is occurring so that a
cache can know when it is semantically transparent to use a particular
cached entity for a future request, and also so that a user agent can
supply more detailed preferences than it might normally send once it
knows they are having an effect on the entity received.

A cache already assumes that response entities will vary based on the
method, identifier, request-entity, and status-code, which is why those
dimensions are not represented in the Vary header field.

Agent-driven negotiation occurs when a client looks at the first
response (the value of response-header fields and/or the response entity)
and decides that a better representation of the first response entity
can be obtained via a GET on some other resource, or, after receipt of 406,
by repeating the original request with a different set of
request-header fields.  Agent-driven negotiation does not interfere
with caches because the two requests are initiated independently.

Transparent negotiation is, essentially, a license for a cache to
act as an agent, on behalf of other agents, for selecting the "better
representation" and initiating pseudo-requests on the second resource
(they are pseudo-requests only in the sense that they may be resolved
internally by another cache hit, and thus no additional network request
is made in that case).  The problem is that, in doing so, they are
performing server-driven negotiation, and thus must add the appropriate
Vary information (if not already present) so that other outbound caches
won't be confused.

There is no easy way to specify this, but ignoring (or disallowing)
any of these cases does not help. It's equivalent to trying to stop
a charging rhino by drawing a line in the sand and trying to explain
to it that we are not ready for it to cross -- we are better-off just
directing the rhino to the safest path (in its own best interest),
and then getting out of the way.

>>>content negotiation
>>>  The mechanism for selecting the appropriate representation when
>>>  servicing a request, as described in section 12.  Note that response
>>>  messages may be negotiated, as well as resources.
>>
>>This last sentence is confusing.  I suggest that this sentence is
>>deleted.

So do I -- it was not in my version.  In any case, the "response message"
is not negotiated -- only the representation of the response entity.

> I think the fact that responses in general are negotiable is
> worty of note.  See if this sentence does better
> 
> Note, therefore that responses of all sorts can be negotiated
> (including error responses), as well as entities derived from
> resources.

I find it more confusing to make special mention of something which
is not a special case -- it causes me to wonder about all the other
non-special cases which are not specifically noted.  I also think
the second note is more confusing than the one I would delete anyway. :)

>>>12 Content Negotiation

>...[other jg changes which look fine to me]
>>>...
> The real bug is the definition of representation, which I don't think
> Roy dealt with properly.  I've reworked the representation definition
> to make clear we mean any entity, not just those of resources.
> 
> representation
>   An entity included with a response that is subject to content
>   negotiation, as described in section 12.  There may exist multiple
>   representations associated with a particular resource.  Likewise,
>   there may be multiple representation of entities returned in error
>   responses.

Nope, I disagree.  It should be just

  representation
    An entity included with a response that is subject to content
    negotiation, as described in section 12.  There may exist multiple
    representations associated with a particular response status.

There always exist multiple representations associated with a particular
resource -- a fact which doesn't imply content negotiation.  The "content"
in content negotiation refers to the content within the response entity,
not to the nature of the resource.

>...[other jg changes which look fine to me]
>>>
>>>12.2 Agent-driven Negotiation
>>>
>>Let me repeat my earlier objection to this section: there is no reason
>>for including this section, as a discussion of transparent negotiation
>>is not needed to justify the HTTP/1.1 protocol elements defined in the
>>pre-04 spec.  The inclusion of this section will only increase the
>>risk that the pre-04 spec is rejected on external review and might
>>unleash a flood of questions on the mailing list that somebody would
>>have to answer.
> 
> I think it very likely the spec will be rejected on this point.

Jim meant to say "unlikely".

> I think people have to see some idea of where we are going,
> to understand the requirements we've laid on them.

Me too.  I am also confident that it won't be rejected without a
specific example which shows it to be broken, and I am fully capable
of defending it against broken examples.

>>>...
>>>Transparent negotiation has the advantage of distributing the
>>>negotiation work that would otherwise be required of the origin server
>>>and also removing the second request delay of agent-driven negotiation
>>>when the cache is able to correctly guess the right response and already
>>                                                              ^^^^^^^^^^^ 
>>>has that response cached.
>> ^^^^^^^^^^^^^^^^^^^^^^^^^
>>
>>In this case, the cache even removes the second request if it does not
>>have the response cached, so the marked text above should be removed.

Okay by me.

> How could the cache help if it doesn't have that response cached?
> Removing the phrase seems to me to reduce the sentence to useless.

When the set of dimensions covered by Alternates matches the set of
dimensions covered by Vary, the cache may interpret the first (general)
GET request and replace it with the more specific resource identified
by the Alternates.  This won't work for some requests (e.g., digest'd ones),
and is not necessarily a good idea, but it is certainly possible.

>>>A cache performing transparent negotiation MUST include the agent-driven
>>>negotiation information along with the response, and MUST add a Vary
>>>header field to the response (defining the dimensions of its variance)
>>>if a Vary field was not already assigned by the origin server.
>>>
>>>These requirements apply to HTTP/1.1 applications even though this
>>>specification does not specify transparent negotiation, since an
>>>understanding of these requirements is a necessary prerequisite for any
>>                                                    ^^^^^^^^^^^^  
>>>future implementation of these features.
>>
>>I read this to mean that HTTP/1.1 applications must be ready to
>>interoperate with _future_ HTTP/1.1 caches which implement transparent
>>negotiation, rather than the other way around.

No, it means that any implementation of transparent negotiation must
obey these requirements in order to avoid breaking HTTP/1.1 caches
that (by that time) will already exist.

>>In my reading, this part of the pre-04 text makes normative reference
>>to a standard that has yet to be defined, which is clearly something
>>that cannot be allowed.

No, it does not.  It states a fact of HTTP (that transparent negotiation
is possible in HTTP/1.1 even though it has not been STANDARDIZED), and
then gives the known requirements associated with that fact.

>>I suggest a complete rewrite of the last two paragraphs of 12.3 to:
>>
>> This specification does not define any mechanism for transparent
>> negotiation, though it also does not prevent any such mechanism from
>> being developed as an extension and used within HTTP/1.1.  A HTTP/1.1
>> cache performing transparent negotiation MUST include a Vary header
>> field in the response (defining the dimensions of its variance) to
>> ensure correct interoperation with all HTTP/1.1 clients.

Which is pretty close to the right thing.

> I see what you are driving at, and talked with Roy about it.
> This is what I came up with afterwards:
> 
> The following requirements apply to HTTP/1.1 applications even though
> this specification does not specify transparent negotiation, to ensure
> that a transparent negotiation extension can be added at a later time
> and used within HTTP/1.1. Any cache that performs transparent
> negotiation SHOULD include the agent-driven negotiation information
> along with the response, and MUST add a Vary header field to the
> response (defining the dimensions of its variance) if a Vary field was
> not already assigned by the origin server, to ensure correct operation
> with all HTTP/1.1 clients.

Ummm, no, that's worse because it conflates the notion of transparent
negotiation (which is defined) and the mechanism for it (which isn't defined).
I see that was what Koen was really objecting to in draft 04a.
Having read all three, I would prefer

  This specification does not define any mechanism for transparent
  negotiation, though it also does not prevent any such mechanism from
  being developed as an extension and used within HTTP/1.1.  An HTTP/1.1
  cache performing transparent negotiation MUST include a Vary header
  field in the response (defining the dimensions of its variance) to
  ensure correct interoperation with all HTTP/1.1 clients.  The
  agent-driven negotiation information supplied by the origin server
  SHOULD be included with the transparently negotiated response.


 ...Roy T. Fielding
    Department of Information & Computer Science    (fielding@ics.uci.edu)
    University of California, Irvine, CA 92717-3425    fax:+1(714)824-4056
    http://www.ics.uci.edu/~fielding/

[any resemblance between existing Internet companies and charging rhinos
 is purely coincidental]

Received on Monday, 3 June 1996 02:05:35 UTC