Re: Striving for Compromise (Consensus?)

On Fri, Jul 11, 2014 at 9:40 PM, Amos Jeffries <squid3@treenet.co.nz> wrote:

> On 12/07/2014 3:57 p.m., Roberto Peon wrote:
> > In cases where HOL blocking is acceptable, you stream.
> > This can be in cases where you know the entity that is sending the
> headers
> > you'd be streaming is trustworthy.
> >
> > If you don't want HOL blocking you don't stream.
> > These don't conflict, they're simply different tradeoffs.
> > -=R
> >
>
> So let me ask you this:
>  Why are you arguing so feircely against a proposal which sends up-front
> the information to identify whether you need to make such a trust
> decision in the first place?
>

That information, about the size of the header, is not trustworthy in a DoS
scenario.

If the information about the supposed size of the header is used to
allocate buffers, it can create an asymmetric DoS scenario, where the
attacker is spending much less resource than the server-side is spending.
One accomplishes this by sending a large header size, and then either never
fulfilling the promise of those bytes. Very little bandwidth gets wasted,
and lots of memory gets consumed.

The obvious work-around is to allocate buffers as they're consumed, which
is essentially what would happen with header fragmentation.

Another issue: Lets say that the server has an internal idea of the amount
of memory it wishes to commit.
It sees that the advertised header size is greater than it is willing to
accept.
It rejects it immediately. It has now leaked information to the attackers
about the state of the server.

And then there is the fact that we do have compression, and one can make
promises till blue in the face about how much memory one is willing to
accept compressed, and it has zero to do with the state commitment
uncompressed if the server has to interpret the headers.

So, the way I look at it, it offers no practical benefit while placing
restrictions on proxies, tunnels and non-interpreting gateways w.r.t.
buffering which induce latency and which increase state commitment even
when they don't interpret. The bad outweighs the good.


> Say you have an implementation that wants to stream. You arbitrarily
> decide to send to your clients a fixed 32KB frame limit, and stream
> anything under 20KB that comes back. When your clients send you new
> HEADERS using Greg et al frames you get told whether it is 16KB, 256
> bytes or 30KB in the frame intro. Make your decision before the first
> byte of headers payload has been processed and take the
> stream/non-stream code path to handle it.
>  How does one do that with HEADERS+CONTNUATION? stream up to 16KB then
> stall? *buffer* a "streamed" frame?
>
>
Why would I set such a limit?
I'd be better off setting a limit on the decompressed size, since that is
more likely to correspond to what the server has to have resident in order
to interpret it.

Best case is ~compression-state-size + compressed frame size.
Worst case is  ~compression-state-size * compressed_frame_size / 2

Even then, why advertise it? It is better to consume the headers, and give
the attackers no direct indication of the effect of the attack.
-=R


> The simplest solution to these and many other issues is what Greg et al
> are proposing.
>




>
> Amos
>
> >
> > On Fri, Jul 11, 2014 at 8:33 PM, Amos Jeffries wrote:
> >
> >> On 12/07/2014 6:30 a.m., Martin Thomson wrote:
> >>> On 11 July 2014 11:08, Willy Tarreau wrote:
> >>>> I think that in Roberto's case, it's the opposite in fact. Assuming he
> >>>> runs a customer-facing gateway that aggregates streams to origin
> servers
> >>>> using less connections, *and* that he forwards headers on the fly
> >> without
> >>>> buffering, then a slow client sending a small header frame and slowly
> >>>> pushing continuation frames would simply block all traffic from all
> >> other
> >>>> clients to that same server for the same connection.
> >>>
> >>> That's true.  So don't do that.
> >>>
> >>> Perhaps we are talking past each other.  Here's the matrix:
> >>
> >> Greg Wrote:
> >>>
> >>> 1) Increase frame size to 16-bits
> >>> 1a) Add a settings for max_frame_size
> >>> 2) Remove reference set from HPACK allowing for "streaming" decoding.
> >>> 3) Requiring that all ":"-headers appear first.
> >>> 4) Only allowing CONTINUATION if the previous frame is max_frame_size.
> >>> 5) Allowing interleaving of CONTINUATION frames with other frames.
> >>> 5b) The size of the HEADERS and CONTINUATION frames are removed from
> the
> >>> flow control window, but the they are never flow controlled.
> >>>
> >>>
> >>>
> >>
> >>
> >>>
> >>> 0 can and do stream (1-3), no need for max_size frames (!4), no
> >>> interleaving (!5)
> >>>
> >>>    - No state commitment DoS concern
> >>>    - May spend extra cycles on framing
> >>>    - May have HOL
> >>>    - No stalling
> >>>
> >>> 1 can and do stream (1-3), need to fill frames before CONTINUATION
> >>> (4), no interleaving (!5)
> >>>
> >>>    - No state commitment DoS concern
> >>>    - No extra cycles on unnecessary framing
> >>>    - May have HOL
> >>>    - May stall
> >>>
> >>> 2 don't stream, no need for max_size frames (!4), no interleaving (!5)
> >>>
> >>>    - This is the status quo
> >>
> >> Strange that Roberto's argument for keeping status-quo was that we need
> >> it to stream headers. But you summarize status-quo under "don't stream".
> >> I find that case (0) above is closer to status-quo.
> >>
> >>>    - No state commitment DoS concern
> >>>    - May spend extra cycles on framing
> >>>    - May have HOL
> >>>    - No stalling
> >>>
> >>> 3 don't stream, need to fill frames before CONTINUATION (4), no
> >>> interleaving (!5)
> >>>
> >>>    - No state commitment DoS concern
> >>>    - No extra cycles on unnecessary framing
> >>>    - May have HOL
> >>>    - May stall
> >>>
> >>> 4 can and do stream (1-3), no need for max_size frames (!4),
> >> interleaving (5)
> >>>
> >>>    - No state commitment DoS concern
> >>>    - May spend extra cycles on framing
> >>>    - Can avoid HOL
> >>>    - May stall
> >>>
> >>>
> >>> 5 can and do stream (1-3), need to fill frames before CONTINUATION
> >>> (4), interleaving (5)
> >>>
> >>>    - State commitment DoS concern
> >>>    - No extra cycles on unnecessary framing
> >>>    - No HOL
> >>>    - May stall
> >>>    - Streaming not possible where max_size doesn't match up
> >>>
> >>> 6 don't stream, no need for max_size frames (!4), interleaving (5)
> >>>
> >>>    - State commitment DoS concern
> >>>    - May spend extra cycles on framing
> >>>    - Can avoid HOL
> >>>    - No stalling
> >>>
> >>> 7 don't stream, need to fill frames before CONTINUATION (4),
> >> interleaving (5)
> >>>
> >>>    - State commitment DoS concern
> >>>    - No extra cycles on unnecessary framing
> >>>    - Can avoid HOL
> >>>    - May stall
> >>>
> >>> I think that's good enough for our purposes.
> >>>
> >>> Given that my priority order here is approximately: state commitment >
> >>> stalling > HOL > CPU, I think that status quo is looking pretty good.
> >>>
> >>> Note that option 5, which is the complete combination of all proposed
> >>> changes, seems to actually be the worst in this analysis.
> >>>
> >>
> >>
> >> Matrix case (2) has more in common with "Greg et al" large frames
> >> proposal than status-quo h2-13.
> >>  * dont stream (large frames requests buffering on sender)
> >>  * no need for max_size frames (large frames sends frame size matching
> >> payload)
> >>  * no interleaving (Greg et al large frames does not permit even
> >> mistaken implementers to accidentally interleave frames inside each
> >> others payload)
> >>
> >> Greg et al proposal causes the '-' bullets to change thusly:
> >>
> >>  - "No state commitment DoS concern" - Greg et al frame limit removes
> >> implementations protocol mandated commitment to handle infinite-length
> >> CONTINUATION payload. Greatlyimproving this over the status-quo.
> >>
> >>  - "May spend extra cycles on framing" - Greg et al frames remove this
> >> issue completely.
> >>
> >>  - "May have HOL" - no change here, but Greg et al frames permit senders
> >> to mediate frame size and prevent HOL blocking on outgoing connections.
> >> status-quo actually mandates buffering "state commitment DoS concern" as
> >> the only possible way to resolve HOL blocking.
> >>
> >>  - "No stalling" - stalling is *always* possible with frames overhead
> >> about 3 bytes. It is a TCP behaviour. h2 has frame overhead 8-12 bytes.
> >>
> >>
> >>
> >> As for your priority order:
> >>
> >>> Given that my priority order here is approximately:
> >>
> >>> state commitment >
> >>
> >> Greg et al. large frames leave the state commitmet equal to status quo
> >> unless implementer decides otherwise. Implemeter may advertise *less*
> >> state commitment than h2 specification default.
> >>
> >>> stalling
> >>
> >> Stalling is possible in all circumstances where more than two bytes need
> >> to be strung together. status quo is *very bad* in encouraging this, as
> >> are the interleaving and fragmentation proposals.
> >>
> >> Greg et al large frames require the sender to accumulate the frame
> >> before delivery to ensure length is accurate. This actively closes many
> >> paths and potential HTTP/2 implementation mistakes leading to stalling.
> >> The remaining causes are all TCP induced and need to be resolved at the
> >> TCP layer.
> >>
> >>> HOL
> >>
> >> Only occurs with status quo and proposals involving CONTINUATION or
> >> fragments.
> >> Greg et al large frames constrains senders not to send at all unless the
> >> frame is fully available and can be delivered.
> >>
> >>
> >>> CPU, I think that status quo is looking pretty good.
> >>
> >> I think that Greg et al frames proposal still looks better.
> >>
> >> Amos
> >>
> >>
> >
>
>