W3C home > Mailing lists > Public > ietf-http-wg@w3.org > July to September 2021

Re: PoW (Re: Attack research on HTTP/2 implementations)

From: Willy Tarreau <w@1wt.eu>
Date: Mon, 6 Sep 2021 18:27:42 +0200
To: Erik Aronesty <erik@q32.com>
Cc: Paul Vixie <paul@redbarn.org>, Nick Harper <ietf@nharper.org>, Erik Nygren <erik@nygren.org>, Stephen Farrell <stephen.farrell@cs.tcd.ie>, Martin Thomson <mt@lowentropy.net>, "ietf-http-wg@w3.org Group" <ietf-http-wg@w3.org>
Message-ID: <20210906162742.GA24506@1wt.eu>
On Mon, Sep 06, 2021 at 10:25:13AM -0400, Erik Aronesty wrote:
> If POW gives small hosts  a 100/1 advantage over not having POW, protecting
> them from all but the largest attacks, this clearly allows smaller hosting
> providers to compete with larger ones in situations where they currently
> cannot.

What you seem to refuse to understand is that it's not just the ratio but
the *limits*, and that if your servers are taken down by small attacks, it
doesn't mean at all that larger attacks are not possible for the attacker,
only that the attacker is sending what's necessary to take your site down.
If you're seeing your server going down with only 10000 connections per
second, you naturally think "ah if I could sustain more that would be
great", except that the client limits itself to your server's
responsiveness, and is barely working above 1%. Even if you would enforce
a 100:1 work ratio it would still be feasible for the client to take your
server down, by giving up on the challenge and trying again.

At the TCP layer, both the attacker and the victim face the very same
limits (in fact slightly less for the attacker who does not need to be
firewalled). This is why I explained that ratios are one thing but they
only apply an offset until the next limit is reached (connection rate
and/or connection count). Without seeking long, I can easily find hosted
machines offering 250 Mbps guaranteed public traffic for $0.07/hr. At
250 Mbps you can set up 186000 TCP connections per second. Good luck
finding a machine accessible to a small company that's able to deal with
such a high connection rate and to deliver TLS PoW challenges! Most of
them will be constrained by their link speed! And I'm not even talking
about the hundreds of thousands or even million concurrent connections
you'll need to sustain just to grant a few seconds timeout before your
regular clients can send their valid requests! And for half a dollar per
hour, that's 750k connections per second that you can send. Some can
even do that from their home fibre connection. I'm not kidding, this
PoW doesn't serve anymore at the packet rates we have to deal with

Another point worth adding, lots of DoS nowadays also involve application
servers. Once the attacker validates the PoW challenge, they can send
countless harmful requests there, that very often thanks to todays lazy
technologies cost way more CPU than the poor 1 ms it takes for an RSA
handshake and cause much more harm. Hint: just loop over the search link
of plenty of sites when it's self-hosted and you'll see that it can takes
*seconds* of CPU, not 1 ms.

As sad as it is, all self-hosted sites nowadays are trivial to instantly
knock down, and it's not due to $bigcorp creating protocols for their own
good but because link speeds increase faster than the processing capacity.
I'm not saying it's not worth defending oneself a little bit, quite the
opposite. All I'm saying is that modifying protocols to include this will
only result in the small ones that cannot update that often to become
harmed even more as attacks will instantly increase 100-fold.

> Obviously it can only be one tool.  But it's still an effective one in many
> scenarios.

It would be effective if it prevented the packets from reaching the server's
external link in the first place.

> I run a relay network where we use this technique after https is
> established on a websocket.

I'm not sure what you mean by "https is established on a websocket". If
you have to setup a websocket connection over HTTPS, the harm was already

> I can tell you that it is defended our network
> admirably against more targeted/smaller attacks dozens of times this year
> alone.

It's only effective for you because most users do not do this by default
and attackers do not waste resources dealing with your specific case. Once
a protection mechanism becomes standard, it's regularly attacked. Look at
SYN floods. A long time ago, SYN cookies were invented to deal with exactly
this. A few years later the protection had been adopted so quickly that
SYN flood rates have reached levels that can barely be handled by dedicated
software or even hardware since their goal has now changed and is now to
saturate links with packets that can difficultly be filtered out.

The best I hope for you if you're using a not-so-common mechanism that
works well for you is that it remains little spread so that it doesn't
become a standard target and continues to work well for you.

Received on Monday, 6 September 2021 16:28:19 UTC

This archive was generated by hypermail 2.4.0 : Monday, 6 September 2021 16:28:20 UTC