- From: James M Snell <jasnell@gmail.com>
- Date: Fri, 11 Jan 2013 10:43:48 -0800
- To: "ietf-http-wg@w3.org" <ietf-http-wg@w3.org>
- Message-ID: <CABP7RbdSurNdLxvVV1F-9Bx4LwdhrX2btEvfA+RV=rVZ0moXNA@mail.gmail.com>
Just continuing my investigation of various header compression strategies
around the BOHE mechanism. In my personal github fork, I have just checked
in two bohe variations, one that implements selective-compression, the
other implements isolated-compression..
https://github.com/jasnell/compression-test/tree/master/compressor/bohe2
https://github.com/jasnell/compression-test/tree/master/compressor/bohe3
With bohe2 (selective-compression), a header block can consist of a
compressed set of headers and an uncompressed set of headers. Specific
headers such as Cookie, Set-Cookie, etc can be marked as "Do Not Compress".
These are dropped into the frame as-is and thus avoid the CRIME issue
completely. The rest of the headers are compressed using gzip using the
existing spdy3 dictionary. Obviously this is not ideal because Cookie data
then is passed around without any compression at all, making it far less
efficient than any of the other options on the table.
In bohe3 (isolated-compression), a header block can consist of two separate
compressed blocks generated using two separate stream compressor instances.
Selected headers (like Cookie) can be included in the secondary isolated
block which would never contain general user-provided header data.
**These are only experiments right now and are not intended as serious
proposals for the spec**. Isolated-compression (bohe3) does show promise
however. If we can successfully isolate potentially sensitive headers into
their own compression context generated independently of any general
user-supplied data, we can effectively short-circuit the CRIME attack by
making it impossible for an attacker to compare values based on the
compression ratio... and since it still uses gzip compression, we achieve a
generally better compression ratio overall than we get with the proposed
delta encoding. For now, tho, just consider this all just to be fodder for
discussion. There are still MANY issues with these experimental approaches
and I still need to go through delta in more detail to see if there is a
way bohe and delta can be used effectively together.
Just for example...
james-snells-macbook-pro:compression-test james$ ./compare_compressors.py
-c bohe3 -c bohe2 -c bohe -c delta -t
/Users/james/git/http_samples/mnot/amazon.com.har
732 req messages processed
compressed | ratio min max std
req bohe 26,035 | 0.13 0.03 0.68 0.08
req bohe2 44,195 | 0.23 0.07 0.71 0.13
req bohe3 30,944 | 0.16 0.05 0.74 0.08
req delta 33,955 | 0.17 0.02 0.71 0.09
req http1 195,386 | 1.00 1.00 1.00 0.00
732 res messages processed
compressed | ratio min max std
res bohe 39,525 | 0.25 0.04 0.67 0.07
res bohe2 47,157 | 0.29 0.12 0.71 0.08
res bohe3 44,843 | 0.28 0.06 0.70 0.07
res delta 44,499 | 0.28 0.02 0.65 0.09
res http1 159,968 | 1.00 1.00 1.00 0.00
- James
On Thu, Jan 10, 2013 at 11:08 AM, James M Snell <jasnell@gmail.com> wrote:
> I have an initial bohe implementation for the compression tests... it's
> very preliminary and uses the same gzip compression as the current spdy3.
> I'm going to be playing around with the delta compression mechanism as well
> and see how much of an impact that has. Initial results are very promising
> but I haven't done much debugging yet. Just wanted folks to know that this
> work was underway...
>
> https://github.com/jasnell/compression-test/tree/master/compressor/bohe
>
> Some test runs....
>
> ./compare_compressors.py -c bohe -c spdy3 -c delta
> ../http_samples/mnot/amazon.com.har
> 732 req messages processed
> compressed | ratio min max std
> req bohe 26,122 | 0.13 0.04 0.70 0.08
> req delta 33,955 | 0.17 0.02 0.71 0.09
> req http1 195,386 | 1.00 1.00 1.00 0.00
> req spdy3 27,238 | 0.14 0.04 0.71 0.08
>
> 732 res messages processed
> compressed | ratio min max std
> res bohe 39,628 | 0.25 0.04 0.66 0.07
> res delta 44,499 | 0.28 0.02 0.65 0.09
> res http1 159,968 | 1.00 1.00 1.00 0.00
> res spdy3 41,325 | 0.26 0.04 0.67 0.08
>
>
> ./compare_compressors.py -c bohe -c spdy3 -c delta
> ../http_samples/mnot/craigslist.org.har
> 66 req messages processed
> compressed | ratio min max std
> req bohe 1,948 | 0.15 0.06 0.73 0.11
> req delta 2,036 | 0.16 0.07 0.71 0.11
> req http1 12,894 | 1.00 1.00 1.00 0.00
> req spdy3 2,016 | 0.16 0.07 0.75 0.11
>
> 66 res messages processed
> compressed | ratio min max std
> res bohe 1,786 | 0.18 0.07 0.77 0.13
> res delta 2,858 | 0.28 0.08 0.69 0.12
> res http1 10,147 | 1.00 1.00 1.00 0.00
> res spdy3 1,869 | 0.18 0.09 0.78 0.13
>
>
> ./compare_compressors.py -c bohe -c spdy3 -c delta
> ../http_samples/mnot/flickr.com.har
> 438 req messages processed
> compressed | ratio min max std
> req bohe 11,988 | 0.10 0.02 0.69 0.07
> req delta 26,372 | 0.22 0.01 0.71 0.14
> req http1 121,854 | 1.00 1.00 1.00 0.00
> req spdy3 12,550 | 0.10 0.02 0.71 0.07
>
> 438 res messages processed
> compressed | ratio min max std
> res bohe 13,073 | 0.09 0.05 0.66 0.06
> res delta 25,236 | 0.18 0.02 0.70 0.11
> res http1 140,457 | 1.00 1.00 1.00 0.00
> res spdy3 14,142 | 0.10 0.05 0.66 0.06
>
>
> ./compare_compressors.py -c bohe -c spdy3 -c delta
> ../http_samples/mnot/facebook.com.har
> 234 req messages processed
> compressed | ratio min max std
> req bohe 6,091 | 0.15 0.06 0.78 0.07
> req delta 7,800 | 0.19 0.02 0.70 0.07
> req http1 41,980 | 1.00 1.00 1.00 0.00
> req spdy3 6,301 | 0.15 0.06 0.77 0.07
>
> 234 res messages processed
> compressed | ratio min max std
> res bohe 9,458 | 0.23 0.07 0.68 0.07
> res delta 12,045 | 0.30 0.13 0.60 0.08
> res http1 40,252 | 1.00 1.00 1.00 0.00
> res spdy3 9,788 | 0.24 0.07 0.69 0.07
>
>
>
>
>
Received on Friday, 11 January 2013 18:44:37 UTC