Re: Transfer-codings, mandatory content-coding support and intermediaries

FWIW, *if* the goal is to provide semantic compatibility between 1.1 and 2.0, and if we envision 2.0 -> 1.1 proxies being used, then I don't think that a gzip bit on each DATA frame is the right approach.

A 2.0 proxy talking to a 1.1 server can fairly easily convert a 2.0 request using a gzip bit into a 1.1 request using T-E: gzip (just concatenate the gzip streams from each DATA frame), but the reverse will require the proxy to decompress the T-E: gzip stream from the 1.1 server and then either provide an uncompressed response to the 2.0 client or re-compress each DATA frame.


On Apr 18, 2014, at 4:50 PM, K.Morgan@iaea.org wrote:

> Despite my comments below, if all we can agree on for this implementation draft is a single gzip bit ("AT RISK"), I think that's better than nothing so we can have some more experience before the discussion in NYC. - As long as something like Matthew's proposal is still on the table (for /2 or /3).
> 
> 
> 
> See my comments in-line below...
> 
> 
> 
> On 18 April 2014 18:38 Martin Thomson wrote:
>> ... we could reasonably mandate the use of gzip on a segment by
>> segment basis, which should help with the security concerns (and
>> provide a better excuse for including END_SEGMENT).  Strategic
>> segmentation could be used to separate content.
> 
> The beauty of Matthew's proposal is the independent compression context for each frame. It considerably reduces the implementation complexity with a relatively negligible loss in compression factor (see [1] for our test results).
> 
> Consider a client managing 100 simultaneous streams. Depending on the interleaving, the client could potentially need to keep around 100 separate decompression contexts. Doing decompression frame-by-frame allows for the possibility of a simple implementation that uses a single context to decompress each compressed frame as it arrives.
> 
> If we go with Matthew's proposal (or similar), I still think the security concerns should be managed with language similar to the following...
> 
>>> Intermediaries MUST NOT add transfer coding(s).  Intermediaries MAY remove transfer coding(s) if necessary. Intermediaries MAY remove and re-apply transfer coding(s), but the transfer coding(s) MUST be re-applied in the same order as applied by the origin.  For example, an intermediary might elect to remove a gzip transfer coding for local storage in its cache in order to more easily extract a range of the content. In this case, when serving a range, the intermediary can choose to send the range with no transfer coding or with gzip transfer coding, but not with any other transfer coding(s).
> 
> 
> 
> There's no way to let intermediaries add compression without some complicated scheme that requires the origin to mark every frame as sensitive or not.
> 
> 
> 
> I am adamant that intermediaries be allowed to remove compression at the transfer layer or we'll have problems transforming 2 -> 1.x.
> 
> I also think intermediaries should be allowed to remove compression and then re-apply it, if desired, for exactly the example case mentioned above which was brought up by Amos (see [2]).
> 
>> Matthew's proposal ... could (and should) be significantly reduced in complexity. ...
>> No alternative compression schemes or negotiation - those lead to interoperability failures.
> 
> To be clear, getting rid of negotation means you don't want Matthew's proposal, you just want a single compression bit.
> 
> 
> 
> I agree it adds complexity, but on the other hand it would be a shame to lock in to gzip considering there are already compression schemes better suited to dynamic compression (i.e. better CPU performance) e.g. LZ4 is ~10-20 faster than gzip (see [3]).
> 
> [1] http://lists.w3.org/Archives/Public/ietf-http-wg/2014AprJun/0156.html
> 
> [2] http://lists.w3.org/Archives/Public/ietf-http-wg/2014AprJun/0115.html
> 
> [3] http://code.google.com/p/lz4/
> 
> 
>> On 18 April 2014 01:03,  <K.Morgan@iaea.org> wrote:
>>> On 18 April 2014 08:52 mnot@mnot.net wrote:
>>>> At this point, it looks like the folks who have asked for TE support in HTTP/2
>>>> are willing to wait on incorporating such a feature into HTTP/2.
>>> 
>>> We're not willing to wait.
>>> 
>>>> the status quo seems like a good path forward ...
>>>> we should discuss this in more depth in NYC.
>>> 
>>> If we're going to discuss this in NYC, then it sounds like a really good reason to include it in the current
> implementation draft - even if it's marked "AT RISK" - so we have experience upon which we can base our discussion.
>>> 
>>> The primary concern surrounding transfer codings (whatever the variety) seems to be security vulnerabilities related to
> intermediaries applying compression without proper context.
>>> 
>>> To appease those concerns, I suggest the following language (or similar language applied to Matthew's frame-level
> transfer coding proposal or the single-bit gzip proposal - we also support Matthew's proposal)...
>>> 
>>> 9.4 Transfer Codings
>>> 
>>> HTTP/2 retains the semantics of Transfer-Encoding with the following additional restrictions and requirements.
>>> 
>>> Servers MUST NOT use the "chunked" transfer coding.
>>> 
>>> Intermediaries MUST NOT add transfer coding(s).  Intermediaries MAY remove transfer coding(s) if necessary.
> Intermediaries MAY remove and re-apply transfer coding(s), but the transfer coding(s) MUST be re-applied in the same order
> as applied by the origin.  For example, an intermediary might elect to remove a gzip transfer coding for local storage in
> its cache in order to more easily extract a range of the content. In this case, when serving a range, the intermediary can
> choose to send the range with no transfer coding or with gzip transfer coding, but not with any other transfer coding(s).
>>> 
>>> 
>>> 
>>> Others have mentioned it is "naiive" to think that intermediaries will keep these rules. Agreed. But it is eqaully
> "naiive" to think intermediares won't do the same thing with C-E. The only solution, IMO, to combat both problems would be
> signed headers (signed by the origin) - a benefit in favor of TE/Transfer-Encoding over frame-level codings, but we would
> be happy with either approach.
>>> 
>>>> However, discussion has also uncovered some problematic aspects of requiring clients to
>>>> support GZIP content-encoding in HTTP/2.
>>>> ...
>>>> That's pretty clearly a major reduction in functionality, and effectively removes
>>>> what we used to call semantic transparency from all HTTP1.x->HTTP2 proxies.
>>> 
>>> +1
>>> 
>>> There were no objections raised by anybody when Matthew, Roland and I brought up these issues, so there doesn't seem to
> be a good reason to keep it for this implementation draft.
>>> 
>>> I still think there needs to be a way to allow servers to compress as much as possible (as brought up by Patrick McManus
> [1] and others [2]).  Implicit gzip on the transfer layer is the right way to accomplish that and retain "semantic
> transparency" because a HTTP1.x -> HTTP2 intermediary can remove transfer codings without consequence.
>>> 
>>> [1] http://lists.w3.org/Archives/Public/ietf-http-wg/2014JanMar/1283.html
>>> [2] https://developers.google.com/speed/articles/use-compression
>>> 
> 
> This email message is intended only for the use of the named recipient. Information contained in this email message and its attachments may be privileged, confidential and protected from disclosure. If you are not the intended recipient, please do not read, copy, use or disclose this communication to others. Also please notify the sender by replying to this message and then delete it from your system.
> 

_________________________________________________________
Michael Sweet, Senior Printing System Engineer, PWG Chair