Re: Review: http://www.ietf.org/id/draft-mbelshe-httpbis-spdy-00.txt

On Wed, Feb 29, 2012 at 11:30 PM, Willy Tarreau <w@1wt.eu> wrote:

> Hi Mike,
>
> On Wed, Feb 29, 2012 at 03:00:58PM -0800, Mike Belshe wrote:
> > Overall, I'm not sure how much to worry about DoS.  In the end, I think
> its
> > something you figure out how to mitigate in implementations - just like
> > we've done it before at every level from DNS to TCP to SSL to HTTP.
> >
> > I don't think its as dire as you've made it out to be.  It's not like
> this
> > amplification attack is difficult to mitigate.  A server wanting to limit
> > it, would probably do the following:
> >
> >    a) First off, because SPDY tries to limit connections to
> one-per-domain,
> > there is a new element of information at the transport layer which can be
> > used for DoS countermeasures.  This needs some experimentation, and I bet
> > it will be useful.
> >
> >    b) Also note that for SPDY over SSL, you've got DoS countermeasures
> from
> > the SSL layer at your disposal.  These are certainly going to slow down
> > many attackers.  (It will also make some sites nervous - especially those
> > that haven't hardened port 443 yet!)
> >
> >    c) If they get through both of those layers, the amplification attack
> is
> > easy to detect, and not a consumption of massive memory, bandwidth, or
> CPU
> > on the server; an implementation looking to conservatively block DoS
> could
> > check for expansions of 2-4KB and probably thwart most everything right
> > there.  Cheap.
>
> I disagree on exactly this point, because in order for the server to know
> that it doesn't want to process this 4 kB request, it first has to
> decompress
> it. Using plain old HTTP, in order for the server to reject a 4 kB request,
> the attacked had to upload it, which would have limited its ability to
> knock
> the server down. Now with just a few bytes an attacker can build a 4 kB
> request
> that the server will have to parse before deciding to reject it. A 4kB
> request
> would fit in 6 bytes from what I've seen, maybe less.


Hi Willy,

Thanks for your thoughtful answer again.  I think we need to look deeper to
settle this.

Let's examine the details of zlib and its format.  I believe there is no
practical amplification attack.  Maybe you'll prove me wrong.

zlib is quite simple.  You can scan it.  You can sanity check it.  You can
verify lz77 dictionary entries and ban any sequences that grow too long
(100B?).  On top of that we're compressing highly structured data, so we
know what the decompressed data must look like.  So if the decompressed
data ever falls out of whack, you can nuke it there too, as you stream it.

Short input sequences will be discardable off the bat - none of this "give
me 6 bytes to make me process 5KB" is going to be possible - you'll have to
send a sizable request to get us to process much out of it.  Then, if
individual dictionary entries grow unreasonably large in the first chunk of
headers (100B?), we'll drop you.  If the expanded name value pairs grow too
large, we'll drop you.  Or if it looks like you're sending junk headers or
unreasonable values, we'll drop you there too.  An attacker might be able
to convince a server to decompress 1000B on a 200B input before detecting
the fraud, but even that seems unlikely if we verify at all the levels I
mentioned already.

For argument's sake, lets be generous and pretend its even a 10x attack on
a 100B input.  Note well that the attack does not scale - even if you can
turn 100 bytes into 1000 bytes processed, it does not mean that you can
send us 10K to get 100K processed.   Because the length caps are going to
be really small, as soon as we see too many bytes, we'll be dropping you,
decreasing your amplification.

All in all, to get our SPDY server to process 1KB of amplified data on your
100B sent, you had to do a full 3 way handshake, plus a full SSL handshake
(with PK op on your *client*, and we got to pick which type of PK op -
8192bit RSA anyone?).  Mice nuts!  You'd be better off attacking with
legitimate requests - most servers will use far more resources trying to
process legit requests than deflating that extraneous 1KB of data.

See what I mean?  Because the size is bounded and because the amplification
is easily capped, the attack is not an attack at all.

Mike






> This means that by
> filling the pipe with 6-bytes requests, a client can make a server parse
> 800
> times that amount of traffic. Filling a pipe at 1 Mbps means the server has
> to process 800 Mbps of requests it will decide to drop in the end. This is
> my precise concern. And more importantly, I doubt that a 4kB limit will
> last
> long. I've used 4kB as a request size limit in haproxy 10 years ago, and I
> quickly had to push it to 8kB to satisfy most sites. In enterprise, it's
> worse, sometimes people configure it up to 32 kB but that's another story.
>
> > Overall, I just don't see how DoS is a silver-bullet against the zlib
> > compressor.
>
> Note that I'm not trying to find a silver buller against zlib, I'm worried
> about zlib making DoS much easier. I have a great respect for all the work
> you have done in optimizing the use of zlib for header compression and I
> don't want to dismiss any of your findings. I just think that one point has
> been overlooked and this point in my opinion is a real problem.
>
> > Almost any new feature has DoS potential (e.g. we haven't even
> > talked about frame flooding or syn_streams yet :-)
>
> Every design choice that makes DoS easier needs to be addressed even if
> it slightly reduces optimizations. DoS is far too common these days, we
> should focus on protecting the internet *before* making it faster for
> mobile users.




>
> Regards,
> Willy
>
>