Re: HTTP 2.0 and a Faster, more Mobile-friendly web

This is more of a generic meta-level comment (and not sure if it has been proposed/discussed before).

When we start comparing implementation choices there are lot of unknowns : browser/OS-and-how-its-tuned/test-cases, clients - their speed, bw and proximity to servers etc. etc and ETC. Its almost impossible (and I'd say futile to compare numbers from tests run in different environments).


I was speaking with Rob earlier in the morning today and we felt that one of things that might be helpful as we do these exercises is : setting up of a common test-bed to perform some of these measurements and experiments. As part of this, we could also document and build  a standard-battery of measurements & scenarios that we want to run through (e.g. Sites with 6 3rd party ads vs. not).

Best,
Rajeev


From: Jitu Padhye <padhye@microsoft.com<mailto:padhye@microsoft.com>>
Date: Tue, 31 Jul 2012 10:56:53 -0700
To: Mike Belshe <mike@belshe.com<mailto:mike@belshe.com>>, Henrik Frystyk Nielsen <henrikn@microsoft.com<mailto:henrikn@microsoft.com>>
Cc: "ietf-http-wg@w3.org<mailto:ietf-http-wg@w3.org>" <ietf-http-wg@w3.org<mailto:ietf-http-wg@w3.org>>, Gabriel Montenegro <Gabriel.Montenegro@microsoft.com<mailto:Gabriel.Montenegro@microsoft.com>>, Rob Trace <Rob.Trace@microsoft.com<mailto:Rob.Trace@microsoft.com>>, "Adalberto Foresti (MS OPEN TECH)" <aforesti@microsoft.com<mailto:aforesti@microsoft.com>>
Subject: RE: HTTP 2.0 and a Faster, more Mobile-friendly web

Mike,

Yes, we do need to explore a larger variety of web pages, including web pages with more resources.

I did not explicitly wipe session state cache between runs. Is there a Chrome switch to do this easily?

Jitu

From: Mike Belshe [mailto:mike@belshe.com]
Sent: Monday, July 30, 2012 9:34 PM
To: Henrik Frystyk Nielsen
Cc: ietf-http-wg@w3.org<mailto:ietf-http-wg@w3.org>; Gabriel Montenegro; Rob Trace; Adalberto Foresti (MS OPEN TECH)
Subject: Re: HTTP 2.0 and a Faster, more Mobile-friendly web

Henrik and team -

Awesome paper!

One thing I notice is that your test pages did not use many resources.  (3 for the dummy web page and 11 for the cnn page).  As you know (http://httparchive.org/trends.php) pages today tend to have ~80 resources per page.  Its the large-resource pages where HTTP slows down, as it can only fetch 6 at a time (2 if you stick to the spec).   It would be interesting to load up a more modern page, like a facebook.com<http://facebook.com> wall page, with dozens of images and resources to really exercise the multiplexing.

The biggest conclusion I read from this is that SSL is hard to make fast (and also very true!) :-)  I'm glad you benchmarked with Chromium because it uses SSL FalseStart to reduce a round trip.  I'm curious if you did anything for certificate validation in this test?  It looks like you're generally seeing a 2RTT overhead of SSL in your tests, which is indicative of the SSL cold-start case.  Can you confirm you wiped the session-ID cache between runs?  It's also indicative of not really exercising the multiplexing, where the SSL overhead gets mitigated as HTTP slows down with 6-at-a-time round trips.

Mike

On Sun, Jul 29, 2012 at 1:14 PM, Henrik Frystyk Nielsen <henrikn@microsoft.com<mailto:henrikn@microsoft.com>> wrote:

Dear All,



We remain committed to the HTTP/2.0 standards process and look forward to seeing many of you this week at the IETF meeting in Vancouver to continue the discussion.  In the spirit of open discussion, we wanted to share some observations in advance of the meeting and share the latest progress from prototyping and testing.



There are currently three different proposals that the group is working through:



   * SPDY (http://tools.ietf.org/html/draft-mbelshe-httpbis-spdy),

   * HTTP Speed+Mobility (http://tools.ietf.org/html/draft-montenegro-httpbis-speed-mobility),

   * Network-Friendly HTTP Upgrade (http://tools.ietf.org/html/draft-tarreau-httpbis-network-friendly).



The good news is that everyone involved wants to make the Web faster, more scalable, more secure, and more mobile-friendly, and each proposal has benefits in different areas that the discussion can choose from.



--- A Genuinely Faster Web ---



The SPDY proposal has been great for raising awareness of Web performance. It takes a “clean slate” approach to improving HTTP.



To compare the performance of SPDY with HTTP/1.1 we have run tests comparing download times of several public web sites using a controlled tested study. The test uses publically available software run with mostly default configurations while applying all the currently available optimizations to HTTP/1.1. You can find a preliminary report on the test results here: http://research.microsoft.com/apps/pubs/?id=170059. The results mirror other data (http://www.guypo.com/technical/not-as-spdy-as-you-thought) that indicate mixed results with SPDY performance.



Our results indicate almost equal performance between SPDY and HTTP/1.1 when one applies all the known optimizations to HTTP/1.1. SPDY’s performance improvements are not consistent and significant. We will continue our testing, and we welcome others to publish their results so that HTTP/2.0 can choose the best changes and deliver the best possible performance and scalability improvements compared to HTTP/1.1.





--- Taking the Best from Each ---



Speed is one of several areas of improvement. Currently, there’s no clear consensus that any one of the proposals is the clear choice or even starting point for HTTP/2.0 (based on our reading the Expressions of Interest and discussions on this mailing list. A good example of this is the vigorous discussion around mandating TLS encryption (http://tools.ietf.org/html/rfc5246) for HTTP/2.0.



We think a good approach for HTTP/2.0 is to take the best solution for each of these areas from each of the proposals.  This approach helps us focus the discussion for each area of the protocol. Of course, this approach would still allow the standard to benefit from the extensive knowledge gained from implementing existing proposals.



We believe that the group can converge on consensus in the following areas, based on our reading of the Expressions of Interest, by starting from the different proposals.



------------------|------------------

Area              | Opinion that

                  | seems to prevail

------------------|------------------

1. Compression    | SPDY or Friendly

------------------|------------------

2. Multiplexing   | SPDY

------------------|------------------

3. Mandatory TLS  | Speed+Mobility

------------------|------------------

4. Negotiation    | Friendly or

                  |   Speed+Mobility

------------------|------------------

5. Client Pull/   | Speed+Mobility

      Server Push |

------------------|------------------

6. Flow Control   | SPDY

------------------|------------------

7. WebSockets     | Speed+Mobility

------------------|------------------



Below, we discuss each HTTP/2.0 element and the current consensus that appears to be forming within the Working Group.



1. Compression



Compression is simple to conceptualize and implement, and it is important. Proxies and other boxes in the middle on today’s Web often face problems with it. The HTTP/2.0 discussion has been rich but with little consensus.

Though some studies suggest that SPDY’s header compression approach shows promise, other studies show this compression to be prohibitively onerous for intermediary devices. More information here would help us make sure we’re making the Web faster and better.



Also, an entire segment of implementers are not interested in compression as defined in SPDY.  That’s a challenge because the latest strawman for the working group charter (http://lists.w3.org/Archives/Public/ietf-http-wg/2012JulSep/0784.html) states that the “resulting specification(s) are expected to be meet these goals for common existing deployments of HTTP; in particular, … intermediation (by proxies, corporate firewalls, ‘reverse’ proxies and Content Delivery Networks).”



We think the SPDY or Friendly proposals is a good starting point for progress.



2. Multiplexing



All three proposals define similar multiplexing models. We haven’t had substantial discussion on the differences. This lack of discussion suggests that there is rough consensus around the SPDY framing for multiplexing.



We think that the SPDY proposal is a good starting point here and best captures the current consensus.



3. Mandating Always On TLS



There is definitely no consensus to mandate TLS for all Web communication, but some major implementers have stated they will not to adopt HTTP/2.0 unless the working group supports a “TLS is mandatory” position. A very preliminary note from the chair (http://lists.w3.org/Archives/Public/ietf-http-wg/2012JulSep/0601.html) states that there is a lack of consensus for mandating TLS.



We think the Speed+Mobility proposal is a good starting point here as it provides options to turn TLS on (or not).



4. Negotiation



Only two of the proposals actually discuss how different endpoints agree to use HTTP/2.0.



(The SPDY proposal does not specify a negotiation method. Current prototype implementations use the TLS-NPN (http://tools.ietf.org/html/draft-agl-tls-nextprotoneg) extension.  While the other proposals use HTTP Upgrade to negotiate HTTP/2.0, some parties have expressed non-support for this method as well.)



We think either of the Friendly or Speed+Mobility proposals is a good starting point because they are the only ones that have any language in this respect.



5. Client Pull and Server Push



There are tradeoffs between a server push model and a client pull model. The main question is how to improve performance while respecting bandwidth and client caches.



Server Push has not had the same level of implementation and experimentation as the other features in SPDY. More information here would help us make sure we’re making the Web faster and better.



We think the Speed+Mobility proposal is a good starting point here, suggesting that this issue may be better served in a separate document rather than tied to the core HTTP/2.0 protocol.



6. Flow Control



There has only been limited discussion in the HTTPbis working group on flow control. Flow Control offers a lot of opportunity make the Web faster as well as to break it; for example, implementations need to figure out how to optimize for opposing goals (like throughput and responsiveness) at the same time.



The current version of the SPDY proposal specifies a flow control message with many settings are that are not well-defined. The Speed+Mobilty proposal has a simplified flow control model based on certain assumptions. More experimentation and information here would help us make sure we’re making the Web faster and better.



We think that the SPDY proposal is a good starting point here.



7. WebSockets



We see support  for aligning HTTP/2.0 with a future version of WebSockets, as suggested in the introduction of the Speed+Mobility proposal.





--- Moving forward ---



We’re excited for the Web to get faster, more stable, and more capable, and HTTP/2.0 is an important part of that.



We believe that bringing together the best elements of the current SPDY, HTTP Speed+Mobility, and Network-Friendly HTTP Upgrade proposals is the best approach to make that happen.



Based on the discussions on the HTTPbis mailing list, we’ve suggested which proposals make the most sense to start from for each of the areas that HTTP/2.0 is addressing. Each of these areas needs more prototyping and experimentation and data. We’re looking forward to the discussion this week.



Sincerely,



Henrik Frystyk Nielsen

Principal Architect, Microsoft Open Technologies, Inc.



Gabriel Montenegro

Principal Software Development Engineer, Microsoft Corporation



Rob Trace

Senior Program Manager Lead, Microsoft Corporation



Adalberto Foresti

Senior Program Manager, Microsoft Open Technologies, Inc.