Re: current HTTP/2 spec prevents gzip of response to "Range" request

On Mar 24, 2014, at 6:50 PM, Mark Nottingham wrote:
> On 25 Mar 2014, at 7:19 am, Roy T. Fielding <fielding@gbiv.com> wrote:
>> On Mar 24, 2014, at 10:04 AM, Martin Thomson wrote:
>> 
>>> On 24 March 2014 09:53, Bjoern Hoehrmann <derhoermi@gmx.net> wrote:
>>>> Section 8.1.3. of the current draft is quite clear that implementations
>>>> must not indicate acceptance of or support for transfer encodings using
>>>> the `TE` header; it seems clear that `Transfer-Encoding: gzip, chunked`
>>>> is not supposed to be used in HTTP/2.0, which is exactly the issue here.
>>> 
>>> That's because Transfer-Encoding is a hop-by-hop thing and the
>>> non-standard "gzip" Transfer-Encoding would need to be understood by
>>> all hops in the chain if it were to be used for ranges.  I'm not aware
>>> of any use of Transfer-Encoding other than "chunked", because it's
>>> virtually impossible to define a new one that interoperates.
>>> 
>>> As I said, I think that this is a confusion between two different but
>>> oft-confused headers:
>>> 
>>> Content-Encoding: gzip
>>> Transfer-Encoding: chunked
>> 
>> It seems you have confused them.  Transfer Encoding is something
>> that can be added or removed by the protocol.  Content Encoding is
>> metadata about the representation.  If the protocol modifies a value
>> for CE, then it breaks anything that relies on the payload being
>> delivered without transformation (e.g., ranges and signature metadata).
>> 
>> If HTTP/2 doesn't allow compression transfer encodings, then it doesn't
>> allow compression by intermediaries.
> 
> ... or at least proxies. Since gateways have a relationship with the origin, they can (and often do) coordinate compression with it.

They can define another protocol, yes, but how would they
handle compression of the request content with standard semantics?
Here we are assuming that the user agent is deciding to compress and
the recipients have to be able to distinguish between content that is
just being compressed for performance of this transfer versus content
that is intended to be forever compressed.

>> I assumed that TE was replaced by
>> a framing mechanism that indicates the payload has been compressed.
>> If that isn't so, then HTTP/2 will be less efficient than HTTP/1 for
>> some use cases (like CDNs).
> 
> It'd be more accurate to say that it's less efficient *in theory*.
> 
> Let's be clear; on the browsing Web, gzip and deflate transfer-codings are basically unused. CDNs do not use them, browsers do not support them, servers do not support them. 
> 
> E.g., <https://issues.apache.org/bugzilla/show_bug.cgi?id=52860>.

That is just a chicken and egg problem.  Dynamic content encoding is
known to be broken in almost every respect, and if I ever have any time
free then that issue will be fixed.

>> And, no, it isn't virtually impossible to introduce standard transfer
>> codings.  It just requires effort by browsers to support one.
>> It also isn't necessary to restrict HTTP/2 features to what a current
>> browser supports.
> 
> That's true. However, transfer-codings other than chunked have resolutely failed to catch on over the past ~15 years. Content-encoding -- with warts and all -- is by far the broadest current practice.

They aren't comparable.  Dynamic CE is deployed, but is known to break
content signatures, ranges, and caches, and is incapable of supporting
selectively compressed transfer from client to server.  Transfer
compression has limited deployment but actually does work when it is
deployed by consenting adults.  You can change that deployment in HTTP/2.
It is well within scope.

> We can certainly talk about re-introducing a flag to indicate that the payload of DATA is compressed. I don't see how we can require it to be used, however, since support for gzip transfer-codings is so poor on the existing Web.
> 
> There are also the security considerations, a la <https://github.com/http2/http2-spec/issues/423>.

I would not require it to be used -- just make it possible to be used
and required to understand (decompress by recipient).  This is what
you already do with HTTP/2 -- just stop calling it content encoding.
It is not the same thing.

FWIW, the section on Use of Compression is incorrect because it assumes
that the channel is secure (and I am not sure why it says plaintext,
because the attack is not limited to text).  What it should say:

"Implementations communicating on a secure channel MUST NOT generate
dynamically compressed content that includes both confidential data
and attacker-controlled data unless separate compression dictionaries
are used for each source of data."

....Roy

Received on Tuesday, 25 March 2014 18:23:34 UTC