RE: END_SEGMENT? (#397)

Daniel's latter point applies to several of the "Intermediaries MUST [NOT]" requirements in the spec - since someone could conceivably build intermediaries sitting on top of server and client APIs and tying them together (and people do, with ours), it's not possible for such an intermediary to comply with some of these unless we propagate framing-layer or compressor-layer knowledge all the way up to the app and back down....

From: Daniel Sommermann [mailto:dcsommer@fb.com]
Sent: Tuesday, June 17, 2014 9:23 AM
To: Jeff Pinner; K.Morgan@iaea.org
Cc: HTTP Working Group; C.Brunhuber@iaea.org
Subject: Re: END_SEGMENT? (#397)

Apologies for the (very) late response on this topic. My high level question here is if this is strictly needed for interop with 1.1. If not, I don't think it warrants the extra complexity and constraints on intermediary behavior.
On 04/17/2014 09:25 AM, Jeff Pinner wrote:

On Thu, Apr 17, 2014 at 12:52 AM, <K.Morgan@iaea.org<mailto:K.Morgan@iaea.org>> wrote:

What is the purpose of END_SEGMENT?


END_SEGMENTS allows the layering of record-based semantics onto the framing layer.

Consider this use case in HTTP/1.1. An API provides a stream of data using HTTP/1.1 chunked encoding. Each chunk contains a single record, and the length of this record is indicated by the chunk length. In HTTP/1.1 the length of these chunks was unconstrained, but when translating into HTTP/2, these chunks must be segmented into multiple data frames to fit within the frame size limit.

If the chunk delineation was meaningful, then END_SEGMENT allows this meaning to be preserved.


I'm not exactly sure how END_SEGMENT alone can be used to preserve chunk semantics. If you are proxying from 2 to chunked 1.1, you would need some other flag to know to add transfer-encoding: chunked and then other flags to know the chunk lengths. Roberto implied in an earlier message that END_SEGMENT is needed to preserve chunk boundaries.


The flag is also useful for layering other record-based protocols on top of HTTP.


Couldn't such protocols include their own delimiters? If a protocol running on HTTP/2 is record based, the record delimiters should be incorporated into the contents of the DATA frame. Otherwise, this seems like an unnecessary layering violation.

I'm curious who plans to use this feature and why END_SEGMENT is the only way to do it. While not exactly a DOS vulnerability for proxies, placing requirements on the framing layer limits how proxies can respond in low memory situations (e.g. a memory-constrained proxy might need to forward a partially received DATA frame as 2 smaller frames). The "MUST NOT coalesce" requirement in particular seems out of place. As with any feature, the complexity is also a potential source of bugs.

The other point brought up is that many HTTP APIs will not expose this segment information, potentially making it difficult to conform to the "intermediaries MUST NOT coalesce" requirement.