Re: #445: Transfer-codings - Alternate proposal

On 9/04/2014 3:37 a.m., K.Morgan@iaea.org wrote:
> Below I've attempted to capture...
> 
> - the problems we are trying to solve with transport codings, and
> 
> - the issues with transport codings brought up in the threads.
> 
> 
> 
> Problems solved with transport codings...
> 
>   a) Allows compression of identity range requests [1-4]
> 
>   b) De-emphasizes on-the-fly C-E & associated issues [2, 5-6]
> 
>   c) Allows extensibility to new/better compression techniques  [7]
> 
>   d) Allows compression by intermediaries (e.g. Roy suggests CDNs could be more efficient this way [8]) (e.g. Amos seems to claim this can be a security benefit by doing hop-specific compression [9])
> 
> 
> 
> Issues with using transport codings...
> 
>   0) Various security issues (e.g. DoS) [10-18].
> 
>   1) Compression at a hop without context about the data. [18-22]
> 
>   2) Requires a frame format change* [23]
> 
>   3) 2-byte tax in every frame* [23]
> 
>   4) Results in a low compression factor if done frame-by-frame* [24]
> 
>   5) Reintroduces hop-by-hop headers** [9, 25-26]
> 
>   6) Poses a problem for adding signed headers in a future rev** [9]
> 
>   7) Implementation complexity [15]
> 
>   8) Disputed benefit of compression by CDNs [17]
> 
>  9) Many seem to simply hate TE/Transfer-Encoding (for whatever reason) [22-23, 27]
> 
> *Specific to Matthew's proposal [28]
> 
> **Specific to our proposal [29]
> 
> 
> 
> Below we suggest two potential solutions to these issues. Other suggested solutions are welcome. (Also, please also let me know if you think I failed to capture any of the issues already brought up.)
> 
> 
> 
> A potential Solution
> 
> - Issues 2-4 are w.r.t. Matthew's proposal so I'll let him address those concerns.
> 
> - We could simply solve issues 0, 1, 5, 6, 8 by adding a special restriction in HTTP/2 on Transfer-Encoding headers disallowing changes at any hop.  Essentially making Transfer-Encoding end-to-end. This is a bit ugly though because it changes semantics and would likely cause confusion. (Matthew also mentioned doing something similar for his proposal, but he didn't get much positive support [30].)
> 
> - I have no answer for issue 9.  Haters are gonna hate.
> 

-1 Downvote from me on this.

Making TE end-to-end effectively makes it a redundant duplicate of CE.

> 
> 
> A potentially better Solution
> 
> - A variation on the above solution would be to introduce a new end-to-end coding called "Message-Encoding".  The key difference between "Content-Encoding" (C-E) and "Message-Encoding" (M-E) would be that C-E comes before the Range is applied and M-E comes after the Range is applied.
> 
>   + M-E would allow seeks (i.e. range requests) on the identity data and allow the partial content to be compressed.
> 
>   + The decision to compress is in exactly the same place as the decision for C-E. The server will be able to decide the most suitable technique for a particular situation (i.e. C-E or M-E).
> 
>   + M-E would also be the better, more natural, choice for on-the-fly compression because range requests wouldn't require the entire content to be compressed just to serve a portion of it.
> 

This seems to be missing the point about Range requests problem.
Part of the bigger problem is that intermediary cachesneed to store the
messages and expect to be able to server sub-ranges out of it to meet
client requests. Whether they received a full object or not this is true.

For example:
 client #1 requests bytes 1000-4000 of a 5KB object at some URI.
 this gets stored by the cache.
 time passes...
 client #2 requests bytes 1500-2000 from the same URI.

 cache should be able to serve that client #2 Range request without
fetching an entirely new message from the origin.

Avoiding re-encoding it for client #2 would be a bonus, but is not
semantically compatible with any HTTP/1 hops in that direction.


>   - The only downside is that intermediaries couldn't add compression, but nobody seems to be fighting strongly for that (ourselves included).
> 

I suspect nobody here is working on satellite link networks. Squid
project is regularly getting requests/pressure from those groups to do
things like force compression, improved caching, etc. anything that
might reduce bandwidth and need to transfer full copies of objects.


> 
> 
> Although I know HTTP/2 isn't allowed to change the existing semantics of HTTP, it's not clear if adding HTTP/2 specific semantics is allowed??
> 

As I understand it we can only add new semantics for cases where there
is no interaction with HTTP/1. ie on the single HTTP/2 hop.


Amos

Received on Tuesday, 8 April 2014 16:35:49 UTC