Re: END_SEGMENT and END_STREAM redundant

On Mon, Apr 21, 2014 at 2:41 AM, David Krauss <potswa@gmail.com> wrote:

> There might be some misunderstanding here… the paragraph beginning “One
> approach would be…” in my earlier message was intended to be separate from
> other suggestions I’ve made. (Sorry there have been so many, but this one
> seems like a good compromise.)
>
> Fleshing out the idea:
>
> 1) The end-segment symbol is transported by an end-segment header.
>
> 2) The end-segment header may be encoded (with an empty or default value)
> by the END_SEGMENT bit. Applications don’t even need to know about this.
> Otherwise it’s just a normal header.
>
>
I believe you're saying that, at the application-layer the application is
notified of the end of a segment by the transformation of an END_SEGMENT
bit in the protocol to an END_SEGMENT header visible to the application?


> 3) The END_SEGMENT bit is decoded after other HPACK decoding, equivalent
> to a literal encoding not added to the header table. (Perhaps I’m missing
> some details here… in implementation terms, it’s simply not part of the
> decoder but tacked on afterward.)
>
>
In practice, regular HPACK encoding of end-segment would just be monkey
> business. The user could give it a value, put it in the reference set, etc.
> It would be weird but I see no ill side-effects. It’s not so different from
> forgoing any other compression method of HPACK.
>

I think you mean that end-segment is represented as a header key with a
special value in the reference set, and thus the index is special/becomes
an opcode.



>
> 4) The end-segment symbol (as treated distinctly from the header) should
> be processed after any other headers or data in the frame, although this is
> only an application-level notion.
>
> 5) If empty HEADERS frames aren’t allowed, an end-segment header alone
> would (optimally) be sent as an empty DATA frame. It should be simple to
> specify that END_SEGMENT doesn’t render a frame “non-empty” per HPACK
> restrictions.
>

They are currently allowed-- with HACK, this would imply the same request
was sent again, at least for HTTP.


>
>
> The reasons to do it are:
>
> 1) General header-oriented APIs can provide the end-segment symbol.
> Programs can make one less API call and avoid adding a special case for the
> special end-segment metadata, which after all isn’t actually special except
> in its wire representation. Of course, this doesn’t preclude having a
> dedicated interface for end-segment as well.
>
> Put simply, “conceptual uniformity and canonicalization.”
>
> 2) Remove the unintended reliability of the framing-layer protocol to
> prohibit application usage. Separate it from the application layer without
> any “thou shalt not” language.
>
> 3) Easy to retrofit onto existing implementations. No new intermediary
> processing is required. An HTTP/2 response maps more cleanly to a sequence
> of HTTP/1.1 responses.
>

Apis could indicate end-of-whatever however they wish, e.g. by the method
you propose here. The current specification would be adaptable to your API
without changes.

Here is some pseudo-code which would implement your API:

# Sets END_SEGMENT on an empty DATA frame if there is no metadata to send,
# else sets END_SEGMENT on a HEADERS frame if there is metadata to send.
SendHeaders(headers):

  if len(headers) == 1 and ":END_SEGMENT" in headers.keys():
    df = NewDataFrame()
    df.SetBit(END_SEGMENT)
    outputStream.append(df)
    return

  hf = NewHeaderFrame()
  for k,v in headers:
    if ":END_SEGMENT" == k:
      hf.setBit(END_SEGMENT)

    hf.appendEncodedHeader(HPACK.encodeHeader(k,v))
  outputStream.append(hf)
-=R



>
>
> On 2014–04–21, at 12:34 PM, Roberto Peon <grmocg@gmail.com> wrote:
>
> An empty HEADERS frame may not actually convey zero metadata, if HPACK is
> being used.
>
> If one wishes to overload HEADERS to signal END_SEGMENT, then one must:
> 1) ensure that applications do not use the same keyspace as the reserved
> key.
>    - can be done, but "smells bad"
> 2) ensure that the compressor state is modified properly to convey *only*
> the key
>    - might hurt a bit, as this essentially renders the compressor
> useless...
> 3) ensure that an empty metadata frame does not convey some meaning.
>    - Not guaranteed today: an empty METADATA frame might convey some
> meaning at the application layer.
>
> END_SEGMENT as a bit avoids these problems.
>
> I agree that we might want more text in there to clarify what should be
> visible to the application (i.e. *not* frames, but rather metadata, data,
> end-segment, end-stream), subject to editor approval.
>
> -=R
>
>
>