Re: HEADERS and flow control

Assuming BLOCKED is there, it works to write it when it would all fit,
though it is ugly.

Lets assume this was being used for something RPC like (e.g. websockets+).
Here is an exploration in more detail.

The alternativesare:
New opcode (e.g. METADATA), flow-controlled.
  (a) uses HPACK format, but allowed zero state
  (b) HPACK context per stream, only interpreted at the eventual endpoint--
proxies do not need to examine this state.
  (c) HPACK context per stream, SETTINGS on how much space is allowed
globally for these

Only write metadata when all of it will fit.
  (d) when blocked send BLOCKED frame
  (e) when blocked stall that stream indefinitely until enough space is
available

(f) Ignore flow control/frame doesn't get flow controlled
(g) Add a new, different setting which sets an upper bound on the total
amount of METADATA allowed to be in flight at once, and require
METADATA_ACK (or equivalent)
(h) Tell people to put each RPC onto its own stream
(i) Tell people to put each RPC onto its own stream, and express dependency
using the prioritization scheme. *Require* relative dependencies to be
expressed through proxies.
(j) Add a new flow control token (e.g. METADATA_WINDOW_UPDATE), which
allows for metadata to be sent. Define a setting for this (potentially).


As a reminder, the use-case here is something RPC and/or WebSocket like,
where we have streams of messages where order may matter.

Option (a) never deadlocks, but it allows no compression.
Option (b) never deadlocks, and allows compression, but requires an
unfortunate amount of memory of a server which wishes to interpret this
metadata.
Option (c) never deadlocks, allows compression, but requires allocations of
more memory at proxies and/or servers, and adds more complexity to
negotiations.
Option (d) never deadlocks, allows compression, but may stall a connection
for some indeterminate period of time.
Option (e) can deadlock, allows compression,
Option (f) never deadlocks but requires potentially infinite amount of
memory consumption in a proxy when a non-malicious actor wishing to make a
bunch of RPCs.
Option (g) never deadlocks, allows compression without increasing memory
footprint, but requires acknowledgement of the receipt of HEADERS/METADATA
from the remote side, which increases chattiness and complexity.
Option (h) never deadlocks, allows compression, but doesn't allow for an
expression of sequencing
Option (i) never deadlocks, allows compression, uses little memory, but
requires more state tracking and makes prioritization a requirement. Ick.
Option (j) never deadlocks, allows compression, uses little memory,
requires tracking one additional counter, slightly increases chattiness,
but arguably little since it would be tacked onto WINDOWS_UPDATE flow
control updates (even if not the same frame).


Option(f) is what would happen today. That seems problematic, and would
make the use of HTTP2 as the basis of an RPC system difficult .

Other options?
-=R



On Fri, May 9, 2014 at 3:55 PM, Johnny Graettinger <
jgraettinger@chromium.org> wrote:

> How would the interaction of stream flow control and continuations be
> managed? A stalled stream may also stall the connection, including control
> frames. Could that also lead to deadlock?
>
>
> On Fri, May 9, 2014 at 6:18 PM, Roberto Peon <grmocg@gmail.com> wrote:
>
>>
>>
>>
>> On Fri, May 9, 2014 at 1:47 PM, Martin Thomson <martin.thomson@gmail.com>wrote:
>>
>>> /me still needs more information.
>>>
>>> This is a change.  The change needs greater justification than "it
>>> might be nice".  So far, that's all I've heard.
>>>
>>
>> I have non-browser customers who are looking at HTTP2 as it stands right
>> now and are pointing out that this will become problematic if HEADERS is
>> actually used as metadata (as it was intended to be) as opposed to simply
>> to create streams. There is no guarantee that the metadata is going to be
>> small. They're mostly willing to swallow the idea that a large block of
>> metadata would (effectively) pause multiplexing for a short time, but
>> worried about the fact that this large metadata actually could account for
>> a fair bit of buffer, and that the product that HTTP2 (etc) would replace
>> did have an operational problem with the lack of this in the past.
>>
>>
>> Hmm.. in scanning the document I think we don't have the requirement in
>> there that HEADERS and DATA have a sequence relationship that must be
>> maintained. We've spoken about it numberous times, but I think we
>> overlooked getting that in there.
>>
>>
>>> I actually think that this is nice.  But nice doesn't cut it for me.
>>>
>>> Given the likelihood that header blocks after the first will be used,
>>> this is just another corner case.  If we use HTTP/1.1 as a guide, the
>>> best analogy there is to chunk extensions and trailers, i.e.,
>>> basically zero use.
>>>
>>> Is the intent to flow control PUSH_PROMISE too?
>>>
>>
>> No-- PUSH_PROMISE is necessary for stream creation. Flow controlling it
>> might engender protocol-induced deadlock.
>>
>>
>>>
>>> (In case you haven't have noticed, I want to finish up here.)
>>>
>>
>> I know. I'll point out I haven't been making trouble unless I have people
>> actually wanting to use the HTTP2 for stuff!
>> Better, the folks who are complaining about this to me are creating
>> implementations now, and as far as I can tell, no-one else out there is.
>> That implies that this change will not slow down interop.
>> -=R
>>
>>
>
>