current HTTP/2 spec prevents gzip of response to "Range" request

Assertion: The HTTP/2 spec, as currently written, prevents gzip compression of the response to a "Range" request, when the client wants a range of the original content (i.e. it is not possible with HTTP/2, but was possible with HTTP/1.1).

Summary: Per RFC2616 (HTTP/1.1), ranges are applied after content encoding.  Thus when a range request is combined with gzip content encoding, an HTTP/1.1 compliant server will return a range of the gzip-compressed representation of the original content, not a gzip-compressed representation of the range of the original content. On the other hand, RFC2616 specifies that transfer encodings are the final codings applied to a message body and allows gzip as a transfer coding.  Thus, the only way to transfer a requested range of bytes with gzip compression on the wire is to use "Transfer-Encoding: gzip".  (Note, the only way a client can indicate to a server that it accepts gzip transfer encoding is by sending the header "TE: gzip".)  Since the HTTP/2 spec, as currently written, disallows transfer codings and "TE" request headers that contain any value other than "trailers", it follows that gzip transfer-encoded responses to range requests are disallowed.

Proposed Actions:
1. The perfect solution, would be to change HTTP/2 Section 9.3 from implicit gzip content encoding to implicit gzip transfer encoding.  Note that a gzip content-encoded response body and a gzip transfer-encoded response body are identical if no range is requested by the client.
2. At a minimum, the value "gzip" needs to be allowed for the "TE" request header and a mechanism needs to be introduced for gzip transfer encoding. This is in the spirit of the WG mandate to maintain the core specifications for HTTP.

Soapbox Statements:
- In our opinion, the common usage of "Content-Encoding: gzip" is actually a misnomer because in most cases it is actually used as a transfer encoding i.e. compressing the data on the wire (e.g. Google seems to heavily use gzip compression on the wire).
- In a limited survey of servers, we found that some deal with the range/gzip problem by either not honouring the range request, or disabling compression for responses with range requests. Other servers strictly follow the spec and provide (unusable?) data. By switching to gzip transfer encoding, this problem would go away.
- We have scenarios where it is necessary to incrementally download growing files (strictly appended). In this case, range requests + gzip compression provide a huge performance gain, but is impossible (in a compliant manner) without using gzip transfer encoding for the reasons mentioned in the above sections.

References:
The HTTP/1.1 spec (RFC2616) requires ranges to apply to the content-encoded content:
- from Section 14.35.1: "Byte range specifications in HTTP apply to the sequence of bytes in the entity-body ..."
- from Section 7.2.1: "entity-body := Content-Encoding( Content-Type( data ) )"

The HTTP/1.1 spec (RFC2616) allows transfer encodings (e.g. gzip) as the final encoding of a response entity-body (see above for definition of entity-body):
- from Section 4.3: "The message-body (if any) of an HTTP message is used to carry the entity-body associated with the request or response. The message-body differs from the entity-body only when a transfer-coding has been applied ...
message-body = entity-body | <entity-body encoded as per Transfer-Encoding>"
- from Section 14.39: "The TE request-header field indicates what extension transfer-codings it is willing to accept in the response ... Examples of its use are: TE: deflate ..."

The HTTP/2 spec disallows transfer encodings and the associated headers:
- from Section 8.1.3: "intermediaries SHOULD also remove other connection-specific header fields, such as ... Transfer-Encoding ..."
- from Section 8.1.3: "...the TE header field, which MAY be present in an HTTP/2 request, but when it is MUST NOT contain any value other than "trailers"..."


Sincerely,
Keith Morgan & Christoph Brunhuber

--
Remote Monitoring Unit
Safeguards, International Atomic Energy Agency (IAEA)
Vienna, Austria


This email message is intended only for the use of the named recipient. Information contained in this email message and its attachments may be privileged, confidential and protected from disclosure. If you are not the intended recipient, please do not read, copy, use or disclose this communication to others. Also please notify the sender by replying to this message and then delete it from your system.