- From: Mark Nottingham <mnot@mnot.net>
- Date: Thu, 17 Jan 2013 12:11:39 +1100
- To: Martin Thomson <martin.thomson@gmail.com>
- Cc: James M Snell <jasnell@gmail.com>, "ietf-http-wg@w3.org" <ietf-http-wg@w3.org>
Getting there, although you may need a small truck to haul the grain of salt that will accompany it... On 17/01/2013, at 12:06 PM, Martin Thomson <martin.thomson@gmail.com> wrote: > James, > > Any feel for how bohe compares on CPU cost? Ideally, the test harness > would provide that information alongside the rest of the data. > > On 11 January 2013 10:43, James M Snell <jasnell@gmail.com> wrote: >> Just continuing my investigation of various header compression strategies >> around the BOHE mechanism. In my personal github fork, I have just checked >> in two bohe variations, one that implements selective-compression, the other >> implements isolated-compression.. >> >> https://github.com/jasnell/compression-test/tree/master/compressor/bohe2 >> https://github.com/jasnell/compression-test/tree/master/compressor/bohe3 >> >> With bohe2 (selective-compression), a header block can consist of a >> compressed set of headers and an uncompressed set of headers. Specific >> headers such as Cookie, Set-Cookie, etc can be marked as "Do Not Compress". >> These are dropped into the frame as-is and thus avoid the CRIME issue >> completely. The rest of the headers are compressed using gzip using the >> existing spdy3 dictionary. Obviously this is not ideal because Cookie data >> then is passed around without any compression at all, making it far less >> efficient than any of the other options on the table. >> >> In bohe3 (isolated-compression), a header block can consist of two separate >> compressed blocks generated using two separate stream compressor instances. >> Selected headers (like Cookie) can be included in the secondary isolated >> block which would never contain general user-provided header data. >> >> **These are only experiments right now and are not intended as serious >> proposals for the spec**. Isolated-compression (bohe3) does show promise >> however. If we can successfully isolate potentially sensitive headers into >> their own compression context generated independently of any general >> user-supplied data, we can effectively short-circuit the CRIME attack by >> making it impossible for an attacker to compare values based on the >> compression ratio... and since it still uses gzip compression, we achieve a >> generally better compression ratio overall than we get with the proposed >> delta encoding. For now, tho, just consider this all just to be fodder for >> discussion. There are still MANY issues with these experimental approaches >> and I still need to go through delta in more detail to see if there is a way >> bohe and delta can be used effectively together. >> >> Just for example... >> >> james-snells-macbook-pro:compression-test james$ ./compare_compressors.py -c >> bohe3 -c bohe2 -c bohe -c delta -t >> /Users/james/git/http_samples/mnot/amazon.com.har >> 732 req messages processed >> compressed | ratio min max std >> req bohe 26,035 | 0.13 0.03 0.68 0.08 >> req bohe2 44,195 | 0.23 0.07 0.71 0.13 >> req bohe3 30,944 | 0.16 0.05 0.74 0.08 >> req delta 33,955 | 0.17 0.02 0.71 0.09 >> req http1 195,386 | 1.00 1.00 1.00 0.00 >> >> 732 res messages processed >> compressed | ratio min max std >> res bohe 39,525 | 0.25 0.04 0.67 0.07 >> res bohe2 47,157 | 0.29 0.12 0.71 0.08 >> res bohe3 44,843 | 0.28 0.06 0.70 0.07 >> res delta 44,499 | 0.28 0.02 0.65 0.09 >> res http1 159,968 | 1.00 1.00 1.00 0.00 >> >> - James >> >> >> >> On Thu, Jan 10, 2013 at 11:08 AM, James M Snell <jasnell@gmail.com> wrote: >>> >>> I have an initial bohe implementation for the compression tests... it's >>> very preliminary and uses the same gzip compression as the current spdy3. >>> I'm going to be playing around with the delta compression mechanism as well >>> and see how much of an impact that has. Initial results are very promising >>> but I haven't done much debugging yet. Just wanted folks to know that this >>> work was underway... >>> >>> https://github.com/jasnell/compression-test/tree/master/compressor/bohe >>> >>> Some test runs.... >>> >>> ./compare_compressors.py -c bohe -c spdy3 -c delta >>> ../http_samples/mnot/amazon.com.har >>> 732 req messages processed >>> compressed | ratio min max std >>> req bohe 26,122 | 0.13 0.04 0.70 0.08 >>> req delta 33,955 | 0.17 0.02 0.71 0.09 >>> req http1 195,386 | 1.00 1.00 1.00 0.00 >>> req spdy3 27,238 | 0.14 0.04 0.71 0.08 >>> >>> 732 res messages processed >>> compressed | ratio min max std >>> res bohe 39,628 | 0.25 0.04 0.66 0.07 >>> res delta 44,499 | 0.28 0.02 0.65 0.09 >>> res http1 159,968 | 1.00 1.00 1.00 0.00 >>> res spdy3 41,325 | 0.26 0.04 0.67 0.08 >>> >>> >>> ./compare_compressors.py -c bohe -c spdy3 -c delta >>> ../http_samples/mnot/craigslist.org.har >>> 66 req messages processed >>> compressed | ratio min max std >>> req bohe 1,948 | 0.15 0.06 0.73 0.11 >>> req delta 2,036 | 0.16 0.07 0.71 0.11 >>> req http1 12,894 | 1.00 1.00 1.00 0.00 >>> req spdy3 2,016 | 0.16 0.07 0.75 0.11 >>> >>> 66 res messages processed >>> compressed | ratio min max std >>> res bohe 1,786 | 0.18 0.07 0.77 0.13 >>> res delta 2,858 | 0.28 0.08 0.69 0.12 >>> res http1 10,147 | 1.00 1.00 1.00 0.00 >>> res spdy3 1,869 | 0.18 0.09 0.78 0.13 >>> >>> >>> ./compare_compressors.py -c bohe -c spdy3 -c delta >>> ../http_samples/mnot/flickr.com.har >>> 438 req messages processed >>> compressed | ratio min max std >>> req bohe 11,988 | 0.10 0.02 0.69 0.07 >>> req delta 26,372 | 0.22 0.01 0.71 0.14 >>> req http1 121,854 | 1.00 1.00 1.00 0.00 >>> req spdy3 12,550 | 0.10 0.02 0.71 0.07 >>> >>> 438 res messages processed >>> compressed | ratio min max std >>> res bohe 13,073 | 0.09 0.05 0.66 0.06 >>> res delta 25,236 | 0.18 0.02 0.70 0.11 >>> res http1 140,457 | 1.00 1.00 1.00 0.00 >>> res spdy3 14,142 | 0.10 0.05 0.66 0.06 >>> >>> >>> ./compare_compressors.py -c bohe -c spdy3 -c delta >>> ../http_samples/mnot/facebook.com.har >>> 234 req messages processed >>> compressed | ratio min max std >>> req bohe 6,091 | 0.15 0.06 0.78 0.07 >>> req delta 7,800 | 0.19 0.02 0.70 0.07 >>> req http1 41,980 | 1.00 1.00 1.00 0.00 >>> req spdy3 6,301 | 0.15 0.06 0.77 0.07 >>> >>> 234 res messages processed >>> compressed | ratio min max std >>> res bohe 9,458 | 0.23 0.07 0.68 0.07 >>> res delta 12,045 | 0.30 0.13 0.60 0.08 >>> res http1 40,252 | 1.00 1.00 1.00 0.00 >>> res spdy3 9,788 | 0.24 0.07 0.69 0.07 >>> >>> >>> >>> >> > -- Mark Nottingham http://www.mnot.net/
Received on Thursday, 17 January 2013 01:12:06 UTC