RE: fyi: An Experimental Study of Web Transport Protocols in Cellular Networks

Hi,

Binoy's not on the list, AFAIK, but I was also working with him in the project.

What Mike says is true. Most of the tests were done about a year ago, with whatever Chrome and the Flip server had at that time, with a lot of help from Mike. I'm not sure if the SPDY specification/implementation has changed considerably in this regard since then. SSL was not included.

What we did was that we setup a copy of the contents of a bunch of real websites on the Flip server supporting SPDY, and accessed then over the mobile network with both SPDY and HTTP. We emulated the website setup also what comes to the use of multiple domains, so that the number of TCP connections used would match real life. We ran the tests primarily on top of GPRS (~100 kbps, 400 ms RTT) and WCDMA HSPA (~1 Mbps, 100 ms RTT). We run a on a real cellular access network and run a large number of runs to avoid random effects. We measured mainly just the time from the first TCP SYN going out to receiving the last byte of content of all objects on the main page, i.e. the full page load time. DNS was not included either.

The results we got were that SPDY was indeed faster than HTTP as expected. It very much depended on the site, but over GPRS most of the sites had 5-15% smaller page load times, while in HSPA it was 10-20% (discluding the initial radio channel setup delay). In GPRS the overall page load times were often in the order of 30-90 seconds, while in HSPA around 10 seconds, so you can calculate the absolute savings from there. Majority of savings seemed to stem from compression, but the better multiplexing of SPDY vs. HTTP also helped on some sites.

There were some unideal cases in our tests. For instance, SPDY still opened as many TCP connections as HTTP, even if it only used one per server. I think that has been fixed since, and would perhaps improve SPDY's figures a little bit. Also the compression gains were not always very intuititive, so there could have been something faulty with that in some cases. So I would not treat these as absolute numbers on how much SPDY is faster than HTTP in mobile networks, but it gives at a least a starting point.

BTW, we also tested the effect of TCP initial window of 3 vs. 10 at the same time. To our amazement, the effect was nearly unnoticeable, less than 5% in majority of cases. The access to the very first objects on the page should benefit more than that, but we didn't measure it.

One conclusion was also that in mobile the best way to make web faster is to upgrade the access networks :) HSPA is superior to GPRS and LTE is even better than that. But also the HTTP/2.0 effort and SPDY would give very useful improvements! So based on our results HTTP/2.0 work clearly makes sense. If we can include the battery saving aspects that Gabriel's draft talks about, even the better. For most people battery is actually more important to speed these days when it comes to smartphones.

Markus


From: ext Mike Belshe [mailto:mike@belshe.com]
Sent: 29 March, 2012 15:13
To: gabriel montenegro
Cc: ietf-http-wg@w3.org
Subject: Re: fyi: An Experimental Study of Web Transport Protocols in Cellular Networks

I worked with Binoy a fair bit when he was getting this setup, as he used software from the spdy codebase extensively.

The data is getting a little old at this point - I believe all of these tests were done over a year ago now.  But Binoy could confirm.

He also didn't test the impact of SSL at all, although his conclusion makes references to SSL, it wasn't actually part of the tested code.  Binoy has confirmed this for me.

Mike

On Thu, Mar 29, 2012 at 11:27 AM, gabriel montenegro <g_e_montenegro@yahoo.com<mailto:g_e_montenegro@yahoo.com>> wrote:
Hi folks,

Just heard of this study which is very relevant to the HTTP 2.0 discussions:

Binoy Chemmagate (Aalto University, Finland)
An Experimental Study of Web Transport Protocols in Cellular Networks
M.S. Thesis, October 2010 - September 2011
http://eggert.org/students/chemmagate-thesis.pdf

Gabriel