Re: Transfer-codings, mandatory content-coding support and intermediaries

On 19/04/2014 10:34 a.m., K.Morgan@iaea.org wrote:
> On 18 April 2014 23:49, msweet@apple.com wrote:
>> ... *if* the goal is to provide semantic compatibility between 1.1
>> and 2.0 ... A 2.0 proxy talking to a 1.1 server can fairly easily
>> convert a 2.0 request using a gzip bit
> 
>> into a 1.1 request using T-E: gzip (just concatenate the gzip
>> streams from each DATA
> 
>> frame), but the reverse will require the proxy to decompress the
>> T-E: gzip stream from the
> 
>> 1.1 server and then either provide an uncompressed response to the
>> 2.0 client or
> 
>> re-compress each DATA frame.
> 
> Or the proxy could simply not declare support (i.e. not send a "TE:
> gzip" header to the server) and then compress the response
> frame-by-frame, but that also seems like a waste of bandwidth and an
> ugly hack.

On the contrary both of those operations are the two Right Ways' (tm) to
do it. Which one is up to the intermediary admin to select

IIRC we seem to have a consensus that a 2.0<->1.1 gateway can do thing
sthat drop the performance of the 1.1 section provided the 2.0 benefits
overall outweighs it.
 So the option of not advertising compression on the 2.0->1.1 hop (which
is almost the status-quo in 1.1) should be acceptible exchange for the
gains from mandatory compression support in the 2.0 traffic arriving at
the gateway.

And since *use* is not mandatory there is not a requirement on the
gateway to encode traffic going in the 1.1->2.0 direction. Although some
may wish to spend some CPU cycles benefiting from the compression.


> 
> Which begs the question, why not TE/Transfer-Encoding? It would
> obviously retain semantic compatibility.
> 
> I don't understand the need to eliminate all hop-to-hop headers
> (mentioned by Martin and others). Specifically, what problems are
> introduced by having T-E in the headers instead of a flag in the
> frame?
> 
> Also, I haven't seen comments yet as to whether or not text similar
> to the following would appease the security concerns (regardless of
> the method; T-E or frame-by-frame) ...
> 
>>>> Intermediaries MUST NOT add transfer coding(s).  Intermediaries
>>>> MAY remove transfer coding(s) if necessary. Intermediaries MAY
>>>> remove and re-apply transfer coding(s), but the transfer
>>>> coding(s) MUST be re-applied in the same order as applied by
>>>> the origin.

NP: this is not good.
 Consider a client advertising X, Z encoding supported. Proxy advertises
X, Y, Z. If the origin chose to layer encoding X-over-Y-over-Z it may
have done so because X-over-Z has a vulnerability. Stripping the X and Y
*both* would seem acceptible, since the client is only hoping for X
and/or Z. But re-encoding X-over-Z is tricky.

IMO the intermediary should either relay as-is or just strip away
encodings down to a layer the client has advertised support for.

>>>>  For example, an intermediary might elect to remove
>>>> a gzip transfer coding for local storage in its cache in order
>>>> to more easily extract a range of the content. In this case,
>>>> when serving a range, the intermediary can choose to send the
>>>> range with no transfer coding or with gzip transfer coding, but
>>>> not with any other transfer coding(s).
> 

-1 on this particular wording. You seem to be conflating
transfer-encoding with content-encoding.

In regards to caching:

* Transfer-Encoding is expected to decode on arrival (or at least act
like it has). It must be uncompressed for delivery to the store logics.
  -> The entity being cached is the un-encoded object form.

* Content-Encoding is expected to retain its compression while in the
storage.
  -> The entity being cached is the encoded object form.


The catch in regards to caching per-segment 2.0 compression is retaining
the SEGMENT tagging on byte ranges within the stream payload. But that
is an implementation detail I am sure we caching people can sort out
fairly easily.

My 2c.:

The need for Transfer-Encoding is explicitly about giving intermediaries
flexibility to choose on the transfer of *specific* payloads. A blanket
all-or-nothing MUST/MUST NOT for every possible payload is too absolute
and inflexible to cover every use case properly.
 Given that, the impetus should be on having a signal from the origin
about whether encoding/compression can be added (or not) by the
intermediaries on a particular segment.

Taking this a little further, I wonder if simply extending
Cache-Control:no-transform to mean that not even Transfer-Encoding is
permitted on this stream is sufficient for resolving the security edge
cases. It should then be okay for the spec to allow Transfer-Encoding on
all transformable traffic by default, same as in HTTP/1.1 today
(perhapse with a must-support for gzip etc which are desirable).
 The per-frame compression should in turn reduce the need for adding
no-transform control in the group of segmented transactions which are
non-cacheable with varying security needs (but if in doubt use
no-transform).

Amos

Received on Saturday, 19 April 2014 06:44:34 UTC