Re: HEADERS and flow control

In RPC use case, client/server can do whatever they want in DATA frames and
any metadata can be included there, which solves ordering problems as
well.  So I am under impression that they don't have to use HEADERS for
this and we don't have to introduce yet another complexity.

Best regards,
Tatsuhiro Tsujikawa
2014/05/10 9:05 "Roberto Peon" <grmocg@gmail.com>:

> Assuming BLOCKED is there, it works to write it when it would all fit,
> though it is ugly.
>
> Lets assume this was being used for something RPC like (e.g. websockets+).
> Here is an exploration in more detail.
>
> The alternativesare:
> New opcode (e.g. METADATA), flow-controlled.
>   (a) uses HPACK format, but allowed zero state
>   (b) HPACK context per stream, only interpreted at the eventual
> endpoint-- proxies do not need to examine this state.
>   (c) HPACK context per stream, SETTINGS on how much space is allowed
> globally for these
>
> Only write metadata when all of it will fit.
>   (d) when blocked send BLOCKED frame
>   (e) when blocked stall that stream indefinitely until enough space is
> available
>
> (f) Ignore flow control/frame doesn't get flow controlled
> (g) Add a new, different setting which sets an upper bound on the total
> amount of METADATA allowed to be in flight at once, and require
> METADATA_ACK (or equivalent)
> (h) Tell people to put each RPC onto its own stream
> (i) Tell people to put each RPC onto its own stream, and express
> dependency using the prioritization scheme. *Require* relative dependencies
> to be expressed through proxies.
> (j) Add a new flow control token (e.g. METADATA_WINDOW_UPDATE), which
> allows for metadata to be sent. Define a setting for this (potentially).
>
>
> As a reminder, the use-case here is something RPC and/or WebSocket like,
> where we have streams of messages where order may matter.
>
> Option (a) never deadlocks, but it allows no compression.
> Option (b) never deadlocks, and allows compression, but requires an
> unfortunate amount of memory of a server which wishes to interpret this
> metadata.
> Option (c) never deadlocks, allows compression, but requires allocations
> of more memory at proxies and/or servers, and adds more complexity to
> negotiations.
> Option (d) never deadlocks, allows compression, but may stall a connection
> for some indeterminate period of time.
> Option (e) can deadlock, allows compression,
> Option (f) never deadlocks but requires potentially infinite amount of
> memory consumption in a proxy when a non-malicious actor wishing to make a
> bunch of RPCs.
> Option (g) never deadlocks, allows compression without increasing memory
> footprint, but requires acknowledgement of the receipt of HEADERS/METADATA
> from the remote side, which increases chattiness and complexity.
> Option (h) never deadlocks, allows compression, but doesn't allow for an
> expression of sequencing
> Option (i) never deadlocks, allows compression, uses little memory, but
> requires more state tracking and makes prioritization a requirement. Ick.
> Option (j) never deadlocks, allows compression, uses little memory,
> requires tracking one additional counter, slightly increases chattiness,
> but arguably little since it would be tacked onto WINDOWS_UPDATE flow
> control updates (even if not the same frame).
>
>
> Option(f) is what would happen today. That seems problematic, and would
> make the use of HTTP2 as the basis of an RPC system difficult .
>
> Other options?
> -=R
>
>
>
> On Fri, May 9, 2014 at 3:55 PM, Johnny Graettinger <
> jgraettinger@chromium.org> wrote:
>
>> How would the interaction of stream flow control and continuations be
>> managed? A stalled stream may also stall the connection, including control
>> frames. Could that also lead to deadlock?
>>
>>
>> On Fri, May 9, 2014 at 6:18 PM, Roberto Peon <grmocg@gmail.com> wrote:
>>
>>>
>>>
>>>
>>> On Fri, May 9, 2014 at 1:47 PM, Martin Thomson <martin.thomson@gmail.com
>>> > wrote:
>>>
>>>> /me still needs more information.
>>>>
>>>> This is a change.  The change needs greater justification than "it
>>>> might be nice".  So far, that's all I've heard.
>>>>
>>>
>>> I have non-browser customers who are looking at HTTP2 as it stands right
>>> now and are pointing out that this will become problematic if HEADERS is
>>> actually used as metadata (as it was intended to be) as opposed to simply
>>> to create streams. There is no guarantee that the metadata is going to be
>>> small. They're mostly willing to swallow the idea that a large block of
>>> metadata would (effectively) pause multiplexing for a short time, but
>>> worried about the fact that this large metadata actually could account for
>>> a fair bit of buffer, and that the product that HTTP2 (etc) would replace
>>> did have an operational problem with the lack of this in the past.
>>>
>>>
>>> Hmm.. in scanning the document I think we don't have the requirement in
>>> there that HEADERS and DATA have a sequence relationship that must be
>>> maintained. We've spoken about it numberous times, but I think we
>>> overlooked getting that in there.
>>>
>>>
>>>> I actually think that this is nice.  But nice doesn't cut it for me.
>>>>
>>>> Given the likelihood that header blocks after the first will be used,
>>>> this is just another corner case.  If we use HTTP/1.1 as a guide, the
>>>> best analogy there is to chunk extensions and trailers, i.e.,
>>>> basically zero use.
>>>>
>>>> Is the intent to flow control PUSH_PROMISE too?
>>>>
>>>
>>> No-- PUSH_PROMISE is necessary for stream creation. Flow controlling it
>>> might engender protocol-induced deadlock.
>>>
>>>
>>>>
>>>> (In case you haven't have noticed, I want to finish up here.)
>>>>
>>>
>>> I know. I'll point out I haven't been making trouble unless I have
>>> people actually wanting to use the HTTP2 for stuff!
>>> Better, the folks who are complaining about this to me are creating
>>> implementations now, and as far as I can tell, no-one else out there is.
>>> That implies that this change will not slow down interop.
>>> -=R
>>>
>>>
>>
>>
>