Re: Last Call: HTTP Extension Framework to Proposed Standard

On Tue, 26 Jan 1999, The IESG wrote:

> 
> The IESG has received a request to consider HTTP Extension Framework
> <draft-frystyk-http-extensions-02.txt> as a Proposed Standard.  This
> has been reviewed in the IETF but is not the product of an IETF Working
> Group.

Dear IESG,

I have just finished looking at draft-frystyk-http-extensions-02.  Not all
of my problems with the 01 version got resolved in this draft.  After some
careful reading I found that protocol elements which confused me when
reading the 01 draft are actually just plain broken.  I have to reverse my
earlier conclusion that the protocol is 'reasonably sound': in fact
draft-frystyk-http-extensions-02 has serious protocol flaws which make it
unfit to go to proposed. 

Overall it looks to me like the Ext: header part of the protocol and
the cachability parts need more work.  The draft is also very short on
advice to implementers and does not have a high-level overview of
protocol interactions before going into specifics.

By allowing for cachable extended responses, the spec opens up a
rather large can of worms.  Below I have identified several places
where the spec does not handle these worms with sufficient care, but
my list is probably not complete.  In my opinion this stuff should be
fixed by avoiding the problem all-together, i.e. by simply making all
extended responses uncachable, and making sure that extended requests
can never be responded to by a plain HTTP/1.x cache.

A detailed list of problems follows.

- Sec 4 last para:

       Optional
       declarations MAY be applied to any HTTP message without any change to
       existing HTTP semantics

This seems to say 'if you apply the declaration to any message, the
HTTP semantics will not change'.  However if extension headers are
applied to a HTTP/1.1 304 message, the message becomes an incorrect
1.1 message which (worse) can cause all kinds of consistency problems
in caches.  Some kind of warning that one cannot piggyback extensions
onto 304 messages is in order, I find the spec quite misleading to
implementers as far as HTTP/1.1 pitfalls are concerned.

- Sec 5 point 3:

          3.  If 2) did not result in a 510 (Not Extended) status code, then
              strip the "M-" prefix from the method name and process the
              remainder of the request according to the semantics of the
              extensions and of the existing HTTP method name as defined in
              HTTP/1.1 [5] or later versions of HTTP.

This implies that a proxy which has processed an end-to-end mandatory
extension should _not_ strip the existing Man header in the request
when forwarding the request towards the origin server (when
'processing the remainder according to the semantics of the existing
HTTP method name as defined in HTTP/1.1').  Not stripping the header
is of course wrong.  If the proxy adds its own M- extension to the
outgoing request without stripping the processed Man header first,
this will cause the ultimate recipient of that extension to process
the old Man extension again.  This will have fun effects especially if
the extension is for a payment protocol.

In the table of section 14 it says 'may strip', which conflicts with
the language above, and is still wrong because it must be MUST strip.

- Sec 5 point 4:

              A server MUST NOT fulfill a request without
              understanding and obeying all mandatory extension
              declaration(s) in a request.

This implies that a request cannot contain two mandatory extensions
which are to be executed by different upstream servers.  For example,
if I want my browser to send a mandatory extension to my firewall
proxy to enable some privacy processing, I cannot at the same time use
a browser plugin which sends a mandatory extension to trigger some
plugin-related functionality in the origin server.  Not being able to
combine two extensions in this way is a rather severe restriction
which kills the whole protocol for me.  It means that plugin vendors
cannot use the protocol for fear of proxies getting in the way, and
that proxy vendors cannot use the protocol for fear of plugin vendors
getting in the way.  Result: nobody can use it.

The Ext: mechanism should be fixed to solve this problem, e.g. the
Ext: header should include the names of the extensions that were
understood and obeyed by different upstream servers.


- Sec 5 para after enumerated list:

       HTTP proxies that
       do not understand the "M-" method name prefix SHOULD return 501 (Not
       Implemented) or turn themselves into a tunnel (see [5]) in which case
       they do not take any part in the communication.

This seems to be a re-statement of a requirement that the HTTP/1.1
spec [5] makes on plain HTTP/1.1 proxies which do not implement
mandatory.  However, as far as I know, this is a particular
interpretation of [5] that is wrong: [5] does not require proxies to
turn themselves into a tunnel, they can forward the request and cache
the response.  The revised 1.1 spec (draft-ietf-http-v11-spec-rev-06)
is a bit more clear on this.

This also has an impact on the table in section 14.

- Sec 5.1 Fulfilling a Mandatory Request

There is no equivalent material on how to fulfill an optional request.
  
- Sec 5.1

       In order to avoid that the Ext header
       field inadvertently is cached in an HTTP/1.1 cache, the response MUST
       contain a no-cache cache-control directive.

This caching discipline is wrong for some extensions, for example
those which `introduce new encodings' (see introduction).  For
example, for an extension which has the same effect as the HTTP/1.1
accept-encoding header, the appropriate caching discipline is

 request:
  M-GET /p/q HTTP/1.1
  Man: "http://www.x.y/accept-encoding"; ns=16-
  16-accept-encoding: gzip, identity;q=0
  ....

 response:
  HTTP/1.1 200 OK
  Ext:
  Vary: man, 16-accept-encoding
  Date: Sun, 25 Oct 1998 08:12:31 GMT
  Expires: Sun, 25 Oct 1998 08:12:31 GMT
  Cache-Control: max-age=100000
  16-content-encoding: gzip
  ....
  [gzip-encoded entity body]

There should be no 'no-cache="ext"', because the extended response
remains valid, the extension remains applied, for future matching
requests.  In fact if we have 'no-cache="ext"', then a cache will
respond with the following if the above request is repeated:

  HTTP/1.1 200 OK
  Vary: man, 16-accept-encoding
  Date: Sun, 25 Oct 1998 08:12:31 GMT
  Expires: Sun, 25 Oct 1998 08:12:31 GMT
  Cache-Control: max-age=100000
  16-content-encoding: gzip
  ....
  [gzip-encoded entity body]

And this will cause the client to conclude that it is dealing with a
'broken server [giving] the false impression that an extended request
was fulfilled' (first para of 5.1. At least I assume this is what the
client is supposed to conclude when Ext is missing. At is nowhere
spelled out explicitly that clients should check for ext: at all!)

- Sec 5.1

       If the
       response is otherwise cachable by HTTP/1.1 caches, the server SHOULD
       include an appropriate max-age cache-control directive:

The unqualified SHOULD here is wrong.  First, servers which are not
the origin server often have not enough information to invent an
`appropriate' max-age value.  Also, adding a max-age can weaken the
cachability of the response in some cases (e.g. when it already had an
expires: <now>, which will lead to problems.

- Sec 7

       The server SHOULD send back all the information necessary for the
       client to issue an extended request. It is outside the scope of this
       specification to specify how the extensions inform the client.

If the spec does not specify how to fulfill a SHOULD, then no client
can be unconditionally compliant until some other spec specifies this.
This is a bit of a problem; the SHOULD should be a MAY.

- Sec 9

       If an extension does
       change the default cachability of the response, the originator MUST
       include a Cache-Control header field containing the cache directives
       corresponding to the desired result of the extended semantics.

The above MUST hands the can of worms I talked about earlier to most
extension implementers.  For a start, just _including_ a Cache-Control
is not sufficient.  A Cache-Control or Expires header which is already
present in the response could weaken the directives in the newly
included Cache-Control, leading to inappropriate caching.  This works
both ways: an included Cache-Control could also weaken already
existing directives.  So it is not that simple, one has to somehow
merge the directives in the old Cache-control and Expires with the new
directives, to produce a set of directives which is stronger than
both.  This can be done, though HTTP/1.1 was not really designed to
support it.  So it will be a _very_ hairy operation, especially if
extension-directives are present.  One could of course take the
obvious way out and put in 'Cache-control: no-store' which is the
strongest directive possible.

Finally, if an extension does changes the default cachability of the
response, it may also be necessary to reflect this by adding/updating
Vary, Expires, and Warning headers.


- Sec 15.1, Vary example

This example was apparently adapted from an e-mail message I wrote.
In that message I noted that correct use of the vary header was
slightly non-trivial.  I intended the 'transform' extension to have a
different effect on the response entity depending on which
transformation was used so my vary header was

  Vary: man, 16-use-transform

but the vary header I see in the draft is

  Vary: man

which is *incorrect* for the extension semantics I had in mind.  The
text next to the example says 'varies on the request extension
declaration' but what I had in mind was 'varies on the request
extension declaration and its associated parameter in its header
field'.

- Sec 15

I have said before (in reviewing the 01 draft) that I am not that
happy with the caching discipline shown in the examples, but I won't
repeat that here because I have already said enough about caching.


Koen.

Received on Monday, 8 February 1999 09:42:01 UTC