Message-Id: <18.104.22.168.19990208094158.0321ad50@localhost> Date: Mon, 08 Feb 1999 09:41:58 -0500 To: firstname.lastname@example.org From: Koen Holtman <Koen.Holtman@cern.ch> (by way of Henrik Frystyk Nielsen <email@example.com>) Subject: Re: Last Call: HTTP Extension Framework to Proposed Standard On Tue, 26 Jan 1999, The IESG wrote: > > The IESG has received a request to consider HTTP Extension Framework > <draft-frystyk-http-extensions-02.txt> as a Proposed Standard. This > has been reviewed in the IETF but is not the product of an IETF Working > Group. Dear IESG, I have just finished looking at draft-frystyk-http-extensions-02. Not all of my problems with the 01 version got resolved in this draft. After some careful reading I found that protocol elements which confused me when reading the 01 draft are actually just plain broken. I have to reverse my earlier conclusion that the protocol is 'reasonably sound': in fact draft-frystyk-http-extensions-02 has serious protocol flaws which make it unfit to go to proposed. Overall it looks to me like the Ext: header part of the protocol and the cachability parts need more work. The draft is also very short on advice to implementers and does not have a high-level overview of protocol interactions before going into specifics. By allowing for cachable extended responses, the spec opens up a rather large can of worms. Below I have identified several places where the spec does not handle these worms with sufficient care, but my list is probably not complete. In my opinion this stuff should be fixed by avoiding the problem all-together, i.e. by simply making all extended responses uncachable, and making sure that extended requests can never be responded to by a plain HTTP/1.x cache. A detailed list of problems follows. - Sec 4 last para: Optional declarations MAY be applied to any HTTP message without any change to existing HTTP semantics This seems to say 'if you apply the declaration to any message, the HTTP semantics will not change'. However if extension headers are applied to a HTTP/1.1 304 message, the message becomes an incorrect 1.1 message which (worse) can cause all kinds of consistency problems in caches. Some kind of warning that one cannot piggyback extensions onto 304 messages is in order, I find the spec quite misleading to implementers as far as HTTP/1.1 pitfalls are concerned. - Sec 5 point 3: 3. If 2) did not result in a 510 (Not Extended) status code, then strip the "M-" prefix from the method name and process the remainder of the request according to the semantics of the extensions and of the existing HTTP method name as defined in HTTP/1.1  or later versions of HTTP. This implies that a proxy which has processed an end-to-end mandatory extension should _not_ strip the existing Man header in the request when forwarding the request towards the origin server (when 'processing the remainder according to the semantics of the existing HTTP method name as defined in HTTP/1.1'). Not stripping the header is of course wrong. If the proxy adds its own M- extension to the outgoing request without stripping the processed Man header first, this will cause the ultimate recipient of that extension to process the old Man extension again. This will have fun effects especially if the extension is for a payment protocol. In the table of section 14 it says 'may strip', which conflicts with the language above, and is still wrong because it must be MUST strip. - Sec 5 point 4: A server MUST NOT fulfill a request without understanding and obeying all mandatory extension declaration(s) in a request. This implies that a request cannot contain two mandatory extensions which are to be executed by different upstream servers. For example, if I want my browser to send a mandatory extension to my firewall proxy to enable some privacy processing, I cannot at the same time use a browser plugin which sends a mandatory extension to trigger some plugin-related functionality in the origin server. Not being able to combine two extensions in this way is a rather severe restriction which kills the whole protocol for me. It means that plugin vendors cannot use the protocol for fear of proxies getting in the way, and that proxy vendors cannot use the protocol for fear of plugin vendors getting in the way. Result: nobody can use it. The Ext: mechanism should be fixed to solve this problem, e.g. the Ext: header should include the names of the extensions that were understood and obeyed by different upstream servers. - Sec 5 para after enumerated list: HTTP proxies that do not understand the "M-" method name prefix SHOULD return 501 (Not Implemented) or turn themselves into a tunnel (see ) in which case they do not take any part in the communication. This seems to be a re-statement of a requirement that the HTTP/1.1 spec  makes on plain HTTP/1.1 proxies which do not implement mandatory. However, as far as I know, this is a particular interpretation of  that is wrong:  does not require proxies to turn themselves into a tunnel, they can forward the request and cache the response. The revised 1.1 spec (draft-ietf-http-v11-spec-rev-06) is a bit more clear on this. This also has an impact on the table in section 14. - Sec 5.1 Fulfilling a Mandatory Request There is no equivalent material on how to fulfill an optional request. - Sec 5.1 In order to avoid that the Ext header field inadvertently is cached in an HTTP/1.1 cache, the response MUST contain a no-cache cache-control directive. This caching discipline is wrong for some extensions, for example those which `introduce new encodings' (see introduction). For example, for an extension which has the same effect as the HTTP/1.1 accept-encoding header, the appropriate caching discipline is request: M-GET /p/q HTTP/1.1 Man: "http://www.x.y/accept-encoding"; ns=16- 16-accept-encoding: gzip, identity;q=0 .... response: HTTP/1.1 200 OK Ext: Vary: man, 16-accept-encoding Date: Sun, 25 Oct 1998 08:12:31 GMT Expires: Sun, 25 Oct 1998 08:12:31 GMT Cache-Control: max-age=100000 16-content-encoding: gzip .... [gzip-encoded entity body] There should be no 'no-cache="ext"', because the extended response remains valid, the extension remains applied, for future matching requests. In fact if we have 'no-cache="ext"', then a cache will respond with the following if the above request is repeated: HTTP/1.1 200 OK Vary: man, 16-accept-encoding Date: Sun, 25 Oct 1998 08:12:31 GMT Expires: Sun, 25 Oct 1998 08:12:31 GMT Cache-Control: max-age=100000 16-content-encoding: gzip .... [gzip-encoded entity body] And this will cause the client to conclude that it is dealing with a 'broken server [giving] the false impression that an extended request was fulfilled' (first para of 5.1. At least I assume this is what the client is supposed to conclude when Ext is missing. At is nowhere spelled out explicitly that clients should check for ext: at all!) - Sec 5.1 If the response is otherwise cachable by HTTP/1.1 caches, the server SHOULD include an appropriate max-age cache-control directive: The unqualified SHOULD here is wrong. First, servers which are not the origin server often have not enough information to invent an `appropriate' max-age value. Also, adding a max-age can weaken the cachability of the response in some cases (e.g. when it already had an expires: <now>, which will lead to problems. - Sec 7 The server SHOULD send back all the information necessary for the client to issue an extended request. It is outside the scope of this specification to specify how the extensions inform the client. If the spec does not specify how to fulfill a SHOULD, then no client can be unconditionally compliant until some other spec specifies this. This is a bit of a problem; the SHOULD should be a MAY. - Sec 9 If an extension does change the default cachability of the response, the originator MUST include a Cache-Control header field containing the cache directives corresponding to the desired result of the extended semantics. The above MUST hands the can of worms I talked about earlier to most extension implementers. For a start, just _including_ a Cache-Control is not sufficient. A Cache-Control or Expires header which is already present in the response could weaken the directives in the newly included Cache-Control, leading to inappropriate caching. This works both ways: an included Cache-Control could also weaken already existing directives. So it is not that simple, one has to somehow merge the directives in the old Cache-control and Expires with the new directives, to produce a set of directives which is stronger than both. This can be done, though HTTP/1.1 was not really designed to support it. So it will be a _very_ hairy operation, especially if extension-directives are present. One could of course take the obvious way out and put in 'Cache-control: no-store' which is the strongest directive possible. Finally, if an extension does changes the default cachability of the response, it may also be necessary to reflect this by adding/updating Vary, Expires, and Warning headers. - Sec 15.1, Vary example This example was apparently adapted from an e-mail message I wrote. In that message I noted that correct use of the vary header was slightly non-trivial. I intended the 'transform' extension to have a different effect on the response entity depending on which transformation was used so my vary header was Vary: man, 16-use-transform but the vary header I see in the draft is Vary: man which is *incorrect* for the extension semantics I had in mind. The text next to the example says 'varies on the request extension declaration' but what I had in mind was 'varies on the request extension declaration and its associated parameter in its header field'. - Sec 15 I have said before (in reviewing the 01 draft) that I am not that happy with the caching discipline shown in the examples, but I won't repeat that here because I have already said enough about caching. Koen.