W3C home > Mailing lists > Public > ietf-http-wg@w3.org > January to March 2013

Re: HTTP/2.0 Magic

From: Mark Nottingham <mnot@mnot.net>
Date: Thu, 21 Feb 2013 17:12:56 +1100
Cc: Willy Tarreau <w@1wt.eu>, "Roy T. Fielding" <fielding@gbiv.com>, HTTP Working Group <ietf-http-wg@w3.org>
Message-Id: <D0B62B1C-AD69-478E-8158-515728864F74@mnot.net>
To: Martin Thomson <martin.thomson@gmail.com>
So, I decided to explore this a bit more. 

I have a list of about 760,000 IP addresses running Web servers, so I put together a quick script to see how they reacted to various magic strings.

For the source, see:
  https://github.com/http2/http2-probe

Below are my notes from some preliminary experimentation today.

Just sending FOO\n\n to ~760,000 servers gives:

FOO\n\n
 641481 CLOSE
  85855 CONN_ERR
  35148 TIMEOUT

"CLOSE" means that the server closed the connection; "CONN_ERR" was various connection errors, 
and "TIMEOUT" means it hangs (5 seconds, in this case).

Let's try adding \r before the \n (all of the following runs are on the TIMEOUTs above).

FOO\r\n\r\n
   1602 CLOSE
    303 CONN_ERR
  33243 TIMEOUT

A bit of help. What if we give it HTTP syntax?

FOO * HTTP/2.0\r\n\r\n
  30082 CLOSE
    396 CONN_ERR
   4670 TIMEOUT

Much better. What if we add Connection: close?

FOO * HTTP/2.0\r\nConnection: close\r\n\r\n
  30310 CLOSE
    337 CONN_ERR
   4501 TIMEOUT

Not significantly better. What if we do Content-Length: 0 instead?

FOO * HTTP/2.0\r\nContent-Length: 0\r\n\r\n
  30072 CLOSE
    369 CONN_ERR
   4707 TIMEOUT

Not helping. What if we add a second "request"?
  
FOO * HTTP/2.0\r\n\r\nBAR * HTTP/2.0\r\n\r\n
  31212 CLOSE
    306 CONN_ERR
   3630 TIMEOUT

Significantly better. Can we save the \r?

FOO * HTTP/2.0\n\nBAR * HTTP/2.0\n\n
  27101 CLOSE
    312 CONN_ERR
   7735 TIMEOUT

Ouch. Does it need the full second request, or just some bits?

FOO * HTTP/2.0\r\n\r\nBAR\r\n\r\n
  31132 CLOSE
    427 CONN_ERR
   3589 TIMEOUT
   
Not bad; our best candidate. About 175 out of the 3589 remaining TIMEOUTs on the do send some text (most often, a HTTP status code), but that's not the majority; most of them well and truly hang.

The distribution of latency on CLOSE (factoring out the round trip) is very heavy towards 0; see  the list at the end (some are negative because of jitter; connect time is used to estimate latency). Note the bumps at 1 and 3 seconds; I suspect these are timeouts in some load balancer or other middlebox, because their Server headers are pretty diverse.

Just to see, what if we truncate the first one too?

FOO\r\n\r\nBAR\r\n\r\n
   1042 CLOSE
     86 CONN_ERR
  11766 TIMEOUT
  
Nope (stopped run when this became obvious). What about some binary?

\xff * HTTP/2.0\r\n\r\n\xff\r\n\r\n
  30271 CLOSE
    342 CONN_ERR
   4535 TIMEOUT
   
Huh. Not as good as text. 



Distribution of CLOSE latency on best candidate:

   4135 -0.00
   6664 0.00
   3579 -0.01
   4243 0.01
   1998 -0.02
   1600 0.02
   1399 -0.03
    664 0.03
    867 -0.04
    445 0.04
    541 -0.05
    398 0.05
    469 -0.06
    322 0.06
    342 -0.07
    280 0.07
    306 -0.08
    175 0.08
    285 -0.09
    178 0.09
    237 -0.10
    227 0.10
    154 -0.11
     80 0.11
    106 -0.12
     42 0.12
     30 -0.13
     39 0.13
     16 -0.14
     38 0.14
      1 -0.15
     37 0.15
      3 -0.16
     44 0.16
      2 -0.17
     37 0.17
     22 0.18
      1 -0.19
     27 0.19
      1 -0.20
     26 0.20
      1 -0.21
     24 0.21
     15 0.22
     16 0.23
      1 -0.24
     30 0.24
      8 0.25
      1 -0.26
     15 0.26
     18 0.27
     11 0.28
      1 -0.29
     10 0.29
      5 0.30
      3 0.31
      1 -0.32
      5 0.32
      2 0.33
      1 0.34
      2 0.35
      6 0.36
      4 0.37
      3 0.38
      4 0.39
      2 -0.40
      5 0.40
      8 0.41
      3 0.42
      1 -0.43
      2 0.43
      4 0.44
      1 -0.45
      3 0.45
      4 0.46
      3 0.47
      1 -0.48
      4 0.48
      9 0.49
      3 0.50
      3 0.51
      3 0.52
      3 0.53
      2 0.54
      1 0.55
      1 0.56
      1 -0.58
      2 0.58
      3 -0.59
      2 0.59
      2 0.60
      3 0.61
      4 0.62
      7 0.63
      1 0.64
      2 -0.65
      1 0.65
      2 0.66
      3 0.68
      2 0.69
      1 -0.70
      2 0.70
      3 0.71
      2 0.72
      2 0.73
      1 0.74
      1 -0.75
      1 0.75
      3 0.76
      2 0.77
      2 0.78
      4 0.79
      2 0.80
      6 0.81
      1 -0.82
      2 0.82
      1 -0.83
      1 0.83
      1 0.84
      2 0.85
      2 0.86
      3 0.88
      2 0.89
      3 0.90
      1 -0.91
      4 0.91
      1 0.92
      1 -0.93
      3 0.93
      2 -0.94
      2 0.94
      1 -0.95
      3 0.95
      2 -0.96
      1 0.96
      6 0.97
      1 -0.98
      1 0.98
      8 -0.99
      1 0.99
     81 -1.00
      6 1.00
      6 -1.01
      1 1.01
      8 -1.02
      2 1.02
      1 -1.03
      2 1.03
      3 1.04
      3 1.05
      2 1.06
      2 1.08
      1 1.09
      2 1.10
      5 1.11
      7 1.12
     30 1.13
      3 1.14
      9 1.15
      3 1.17
      3 1.18
      2 1.19
      9 1.20
      1 1.21
      4 1.23
      2 1.26
      5 1.27
      3 1.29
      2 1.31
      1 1.32
      1 1.37
      2 1.39
      1 1.40
      1 1.48
      1 1.49
      1 1.52
      1 1.55
      2 1.56
      1 1.58
      1 1.59
      2 1.60
      1 1.61
      2 1.62
      2 1.63
      1 1.64
      1 1.65
      1 1.67
      2 1.68
      2 1.71
      1 1.73
      1 1.75
      1 1.79
      1 1.81
      2 1.83
      1 1.85
      2 1.87
      1 1.88
      1 -1.97
      1 2.01
      1 2.02
      1 2.03
      1 2.06
      1 2.09
      1 2.12
      1 2.17
      1 2.19
      1 2.20
      1 2.21
      2 2.22
      2 2.24
      2 2.25
      1 2.29
      1 2.37
      2 2.40
      1 2.45
      1 2.46
      1 -2.47
      1 2.47
      2 2.48
      2 2.53
      2 -2.55
      1 2.56
      1 2.60
      1 2.62
      1 -2.67
      2 2.67
      1 2.69
      1 -2.70
      1 2.72
      1 2.81
      1 2.82
      1 -2.86
      1 2.87
      1 2.88
      1 -2.89
      2 -2.90
      1 -2.91
      1 -2.92
      1 2.93
      4 -2.94
     10 -2.95
      2 -2.96
      2 2.96
     11 -2.97
      8 -2.98
      2 2.98
     10 -2.99
      4 2.99
    127 -3.00
     70 -3.01
      1 -3.02
      1 -3.03
      1 3.03
      1 3.04
      1 -3.05
      1 3.06
      1 -3.14
      1 3.15
      1 3.16
      1 3.21
      1 -3.23
      1 3.23
      1 3.24
      1 -3.26
      1 3.32
      1 3.34
      2 3.39
      1 3.46
      1 3.50
      1 3.59
      1 3.81
      1 3.84
      1 -4.00
      1 4.09
      1 4.15
      1 4.34
      1 4.46
      1 4.48
      2 4.49
      1 4.50
      1 4.52
      1 4.53
      2 4.54
      1 4.56
      2 4.57
      3 4.59
      6 4.60
      1 4.61
      1 4.62
      1 4.63
      1 4.72
      2 4.75
      1 4.85
      1 4.91
      1 4.93
      4 4.97
      2 4.98
     14 4.99
     18 5.00
     10 5.01
      5 5.02
      2 5.03
      2 5.04
      2 5.05
      2 5.06
      2 5.07
      1 5.08
      2 5.10
      1 5.12
      1 5.13
      1 5.15
      3 5.18
      1 5.23
      2 5.24
      2 5.29
      1 5.35
      1 5.40
      1 5.43
      1 5.47
      1 5.48
      1 5.50
      1 5.51
      1 5.65
      1 5.79
      1 5.80
      1 5.91
      1 5.97
      1 6.00
      1 6.38
      1 6.43
      1 6.53


Cheers,



--
Mark Nottingham   http://www.mnot.net/
Received on Thursday, 21 February 2013 06:13:25 GMT

This archive was generated by hypermail 2.2.0+W3C-0.50 : Thursday, 21 February 2013 06:13:32 GMT