Re: #445: Transfer-codings

On 5/04/2014 11:18 a.m., K.Morgan@iaea.org wrote:
> See responses in-line below...
> 
> 
> 
> On 04 April 2014 23:20, msweet@apple.com wrote:
>> The origin server sets Content-Encoding and is able to make an
>> informed decision about what content is compressed.  Intermediates
>> are not supposed to change that.  But they *can* change
>> Transfer-Encoding while still not knowing whether the content
>> should be compressed.
>> 
> On 04 April 2014 22:41, Martin Thomson Wrote:
>> On 4 April 2014 13:36,  <K.Morgan@iaea.org> wrote:
>>> If there are specific security issues related to gzip then they
>>> apply equally to C-E gzip (which is implicitly required for
>>> clients to support already). As we mentioned below, one of the
>>> benefits of using TE/Transfer-Encoding over a framing layer
>>> transfer coding mechanism is that the decision to TE compress can
>>> be made at the same level as deciding to CE compress.
>> 
>> This is not correct.  An origin server applies Content-Encoding. 
>> Transfer-Encoding is hop-by-hop.  As I pointed out previously, the 
>> entities applying T-E are likely to lack the contextual
>> information required to make the right decisions regarding safe use
>> of compression.
> 
> In response to both of you, I am either totally missing something or
> these security concerns related to an intermediary adding compression
> without contextual information are dubious.
> 
> The security concerns all relate to compression within a TLS/SSL
> tunnel, but:
> 1) In a secure tunnel, the origin server is the only one
> that can apply a transfer encoding.  Intermediaries only see
> encrypted content and *can't change the transfer coding*.

This is incorrect. It assumes that the secure tunnel is end-to-end
between user agent and origin. Tunnels may only exist as secured hops
which are beyond the knowledge of the origin AND user agent.
 This is a common scenario for Squid proxies tunneling between corporate
POP.

In such cases the endpoints could choose CE compression and unknowingly
cause the traffic to face all the very same vulnerabilities people are
being so worried about for TE.
With TE present the proxy endpoints for the tunnel are in a position to
remove compression (or not) as applicable to the security needs of that hop.


> 2) In the
> case of plain-text communication it doesn't matter if the
> intermediary adds or removes a transfer encoding.
> 
> The framing layer proposal, however, suffers from this security
> problem, even at the origin server, because the framing layer is
> likely unaware of the necessary contextual information.
> 

Which contextual information exactly? I've been following these threads
and only seen mention of issues compressing values from *different*
headers or requests using the same context causing actual crossover
vulnerabilities.

 There is a bunch of contextual information delivered in the HEADERS
which the intermediaries are very well able to be aware of. For example;
Content-Type/Content-Encoding to prevent double-encoding, and the URI
scheme (if not the transport layer itself) to prevent compression over TLS.

 As mentioned above there are hop-specific security contexts which the
endpoints are very much unaware of that may cause the CE vs TE security
decision to backfire.

Locking the protocol into a end-to-end decision on the assumption that
not all implementations will get security right seems to be a bad idea.
Make each *able* to secure easily with encouragement to do so properly.



> On 04 April 2014 22:23, Martin Thomson wrote:
>> On 4 April 2014 11:41, K.Morgan wrote:
>>> The proposals by Matthew and Mark get the job done, but they
>>> strike us as more complex than TE/Transfer-Encoding...
>> 
>> I don't agree.  Both are considerably simpler.
> 
> I disagree with your disagreement :)
> 
> Implementation requirements of TE/Transfer-Coding... Servers: None if
> they don't want to support TE. To support TE: logic in parallel with
> the existing C-E logic to decide which type to use (i.e. whether to
> send the C-E or TE header) - use the same compression code as for
> C-E Clients: if (is_content_encoding_gzip ||
> is_transfer_encoding_gzip) decompress();
> 
> Implementation requirements of framing layer transfer codings: 
> Servers: At a minimum support a new setting. To fully support
> transfer coding: support new setting (including settings table for
> acceptable encodings); add logic at the framing layer to decide if
> content should be compressed; add flag and encoding bytes for each
> data frame Clients: support a new setting; add parsing logic for
> flag/encoding; add logic at the framing layer to decompress content
> if necessary; BTW, as the framing layer proposal now stands, the
> transfer coding compression can go in either direction so both sides
> would require the extra logic above for servers/clients.

In the framing proposal each intermediary endpoint can arbitrarily
decode and re-encode individual DATA frames using an encoding suitable
for the hop it is being transfered over. The only requirement is that
the encoding be accepted by the recipient.


> 
> On 04 April 2014 22:23, Martin Thomson wrote:
>> ...we've removed hop-by-hop headers from HTTP/2.  I don't think 
>> that re-introducing them would be a good idea.
> 
> Why not? (Please be more specific.) I get why the other Connection
> hop-by-hop headers don't make sense, but why are transfer-encoding
> headers such a bad idea? We gave a list of 5+ benefits of using
> TE/Transfer-Encoding.

Using hop-by-hop headers would require one of two things:

 1) if present in the HEADERS/CONTINUATION frames. That would make it
mandatory to implement HEADERS decoding and re-encoding at each hop.
Including in blind relays which would otherwise just handoff the frames
untouched.

 2) if a new type of frame to be added just for these headers. That is a
blocking change on the draft while we re-hash all the decisions made
about HEADERS.

Hop-by-hop details are framing layer details in any event and mixing
them with headers again is a step backwards in efficiency.

#1 would also prevent adoption of security features such as signed
HEADERS frames which might be used on an HTTP/2 end-to-end connection to
verify no transformations are taking place. The present draft leaves
that option open for future use.


Amos

Received on Saturday, 5 April 2014 07:38:39 UTC