Re: current HTTP/2 spec prevents gzip of response to "Range" request

On Mon, Mar 24, 2014 at 9:51 AM,  <K.Morgan@iaea.org> wrote:
> Assertion: The HTTP/2 spec, as currently written, prevents gzip compression
> of the response to a "Range" request, when the client wants a range of the
> original content (i.e. it is not possible with HTTP/2, but was possible with
> HTTP/1.1).
>
>
>
> Summary: Per RFC2616 (HTTP/1.1), ranges are applied after content encoding.
> Thus when a range request is combined with gzip content encoding, an
> HTTP/1.1 compliant server will return a range of the gzip-compressed
> representation of the original content, not a gzip-compressed representation
> of the range of the original content. On the other hand, RFC2616 specifies
> that transfer encodings are the final codings applied to a message body and
> allows gzip as a transfer coding.  Thus, the only way to transfer a
> requested range of bytes with gzip compression on the wire is to use
> "Transfer-Encoding: gzip".  (Note, the only way a client can indicate to a
> server that it accepts gzip transfer encoding is by sending the header "TE:
> gzip".)  Since the HTTP/2 spec, as currently written, disallows transfer
> codings and "TE" request headers that contain any value other than
> "trailers", it follows that gzip transfer-encoded responses to range
> requests are disallowed.
>
>
>
> Proposed Actions:
>
> 1. The perfect solution, would be to change HTTP/2 Section 9.3 from implicit
> gzip content encoding to implicit gzip transfer encoding.  Note that a gzip
> content-encoded response body and a gzip transfer-encoded response body are
> identical if no range is requested by the client.
>
> 2. At a minimum, the value "gzip" needs to be allowed for the "TE" request
> header and a mechanism needs to be introduced for gzip transfer encoding.
> This is in the spirit of the WG mandate to maintain the core specifications
> for HTTP.
>
>
>
> Soapbox Statements:
>
> - In our opinion, the common usage of "Content-Encoding: gzip" is actually a
> misnomer because in most cases it is actually used as a transfer encoding
> i.e. compressing the data on the wire (e.g. Google seems to heavily use gzip
> compression on the wire).
>
> - In a limited survey of servers, we found that some deal with the
> range/gzip problem by either not honouring the range request, or disabling
> compression for responses with range requests. Other servers strictly follow
> the spec and provide (unusable?) data. By switching to gzip transfer
> encoding, this problem would go away.
>
> - We have scenarios where it is necessary to incrementally download growing

For the sake of offering a defense -

This use case is probably not blessed by the letter of the spec:

http://tools.ietf.org/html/draft-ietf-httpbis-p5-range-26#section-4.3

  "These ranges can only be safely combined if they all have in common the
   same strong validator"

   "A client that has received multiple partial responses to GET requests
   on a target resource MAY combine those responses into a larger
   continuous range if they share the same strong validator."

Also, multipart/byteranges may carry a gzip-ed range, though
multipart/byteranges is probably not well supported.

Zhong Yu


> files (strictly appended). In this case, range requests + gzip compression
> provide a huge performance gain, but is impossible (in a compliant manner)
> without using gzip transfer encoding for the reasons mentioned in the above
> sections.
>
>
>
> References:
>
> The HTTP/1.1 spec (RFC2616) requires ranges to apply to the content-encoded
> content:
>
> - from Section 14.35.1: "Byte range specifications in HTTP apply to the
> sequence of bytes in the entity-body ..."
>
> - from Section 7.2.1: "entity-body := Content-Encoding( Content-Type( data )
> )"
>
>
>
> The HTTP/1.1 spec (RFC2616) allows transfer encodings (e.g. gzip) as the
> final encoding of a response entity-body (see above for definition of
> entity-body):
>
> - from Section 4.3: "The message-body (if any) of an HTTP message is used to
> carry the entity-body associated with the request or response. The
> message-body differs from the entity-body only when a transfer-coding has
> been applied ...
>
> message-body = entity-body | <entity-body encoded as per Transfer-Encoding>"
>
> - from Section 14.39: "The TE request-header field indicates what extension
> transfer-codings it is willing to accept in the response ... Examples of its
> use are: TE: deflate ..."
>
>
>
> The HTTP/2 spec disallows transfer encodings and the associated headers:
>
> - from Section 8.1.3: "intermediaries SHOULD also remove other
> connection-specific header fields, such as ... Transfer-Encoding ..."
>
> - from Section 8.1.3: "...the TE header field, which MAY be present in an
> HTTP/2 request, but when it is MUST NOT contain any value other than
> "trailers"..."
>
>
>
>
>
> Sincerely,
>
> Keith Morgan & Christoph Brunhuber
>
>
>
> --
>
> Remote Monitoring Unit
>
> Safeguards, International Atomic Energy Agency (IAEA)
>
> Vienna, Austria
>
>
>
> This email message is intended only for the use of the named recipient.
> Information contained in this email message and its attachments may be
> privileged, confidential and protected from disclosure. If you are not the
> intended recipient, please do not read, copy, use or disclose this
> communication to others. Also please notify the sender by replying to this
> message and then delete it from your system.

Received on Monday, 24 March 2014 15:29:47 UTC