W3C home > Mailing lists > Public > ietf-http-wg@w3.org > April to June 2012

Re: multiplexing -- don't do it

From: Jon Leighton <leighton@cis.udel.edu>
Date: Thu, 05 Apr 2012 14:18:30 -0400
Message-ID: <4F7DE1F6.3060605@cis.udel.edu>
To: ietf-http-wg@w3.org
On 04/05/2012 12:43 PM, Peter Lepeska wrote:
> What's the main reason for the benefit? If it's just that SCTP is less 
> of a good citizen in how it handles a lost packet and backs off slower 
> than TCP then it's not something to get excited about. Have you done 
> much analysis to explore the reason for SCTP being faster?

Although there's some variation in the details between the different 
implementations on different OSes, and there are various sysctl knobs 
you can turn, SCTP's congestion control mechanism is essentially the 
same as TCP's. I'm not certain, but I think it was a design goal to 
ensure that SCTP was no more aggressive than TCP. It's not fundamentally 
any more or less a good citizen.

As Mike already mentioned, I think the reason SCTP does better with loss 
is that it avoids a lot of HOL blocking. Getting HOL blocked on a 
response that's going to cause the browser to open a new connection is a 
problem, and there was a lot of opportunity for that problem given that 
most of the test pages opened 10 or more connections and several opened 
more than 30. I don't know how much SCTP benefited from the random loss 
model. Certainly using a more realistic loss model would improve the 
results.

> Peter
>
> On Thu, Apr 5, 2012 at 11:00 AM, Jon Leighton <leighton@cis.udel.edu 
> <mailto:leighton@cis.udel.edu>> wrote:
>
>     On 04/04/2012 06:05 PM, Mike Belshe wrote:
>>
>>
>>     On Wed, Apr 4, 2012 at 1:58 PM, Peter Lepeska
>>     <bizzbyster@gmail.com <mailto:bizzbyster@gmail.com>> wrote:
>>
>>         As long as SPDY is sent over TCP, it also suffers from HOL
>>         problems, just not as bad as pipelining.
>>
>>         I think SPDY (or whatever the HTTP 2.0 muxing protocol is)
>>         should framed in such a way that if running over a protocol
>>         like SCTP, that solves the HOL problems, we should be able to
>>         take advantage of it. Due to gzip compression of headers,
>>         even if the transport allowed me to grab messages out of
>>         order, I'd still have to wait for all packets prior in order
>>         to decode the HTTP headers.
>>
>>
>>
>>     I think people have confusion about layering on top of transport
>>     protocols.  Any time you have an app protocol that wants to take
>>     advantage of new transport features you *MUST* change the
>>     definition of how the app protocol is bound to the lower level
>>     transport.  This absolutely applies to HTTP and TCP/SCTP (not
>>     talking about SPDY yet).  For example, RFC2616 does *not* specify
>>     how to use HTTP over SCTP, and a whole I-D exists for that:
>>     http://tools.ietf.org/html/draft-natarajan-http-over-sctp-01
>>      This I-D defines one possible binding between HTTP and SCTP, but
>>     others could exist too.
>>
>>     Why is this necessary?  Well, TCP has a single, bidirectional
>>     stream.  SCTP doesn't have bi-directional streams at all, and
>>     instead only has multiple, uni-directional streams.  So, if you
>>     want to leverage the new feature (streams) you need to define how
>>     you're going to bind onto it.  It's trivial, but doable.
>>
>>     If we had SCTP, we wouldn't do MUX and SPDY.  It's redundant.
>>      That's not to say that HTTP/2.0 is worse with multiplexing, its
>>     just to say that we wouldn't need to consider it if the transport
>>     already had it.  But SCTP is not viable today or even within the
>>     next decade.
>>
>>     Google sponsored a project at the Univ of Delaware where they
>>     investigated SPDY over SCTP already.  A couple of bindings for
>>     SPDY over SCTP were considered:  use stream 0 for a control
>>     stream, and send the headers over that stream.  This has the nice
>>     property that it binds the stateful compression to a single,
>>     in-order stream.  Also, as you can imagine, it introduces a small
>>     amount of HoL there.  Another solution was to use N SCTP streams,
>>     with M mux'd SPDY streams on top of it.  Other such bindings
>>     could be considered.  Overall, SCTP was too immature to really
>>     benchmark.  The FreeBSD and Linux implementations of SCTP have
>>     many problems already.  The UoD guys might want to comment, I
>>     found the whole thing non-conclusive.
>
>     For anyone interested, here's a link to a summary of the work
>     http://www.eecis.udel.edu/~leighton/SPDY.html
>     <http://www.eecis.udel.edu/%7Eleighton/SPDY.html>. Under the
>     conditions tested, SPDY over SCTP was clearly faster than SPDY
>     over TCP. This is true regardless of which mapping was used.
>     Though the data is not provided in the summary, using SCTP stream
>     0 as a control stream for SPDY headers was only very slightly
>     faster than sending headers on the same SCTP stream as the data.
>     There were some issues with the linux implementation of SCTP that
>     had to be worked around (most since fixed), but I can't comment on
>     any issues with SCTP on FreeBSD as we decided against trying to
>     port Chrome and the flip_in_mem_edsm_server to FreeBSD. The
>     results were inconclusive with regard to how best to map SPDY to
>     SCTP. Regarding the benefit of using SCTP to avoid HOL blocking,
>     the results are pretty straightforward - SPDY performed better
>     over SCTP than TCP. Increasing the delay and loss increases SCTP's
>     advantage.
>
>
>>     My recommendation:
>>     Designing HTTP/2.0 for SCTP is a mistake and should NOT be a
>>     requirement.  SCTP is not a viable transport over the Internet
>>     today, and will not be in the foreseeable future.  When it is
>>     available, an appropriate binding for HTTP/2.0 can be determined,
>>     trivially, and we can worry about it then.  This is the same
>>     approach that was taken for HTTP with
>>     http://tools.ietf.org/html/draft-natarajan-http-over-sctp-01.
>>
>>     Mike
>>
>>
>>
>>         Peter
>>
>>
>>         On Wed, Apr 4, 2012 at 9:07 AM, Patrick McManus
>>         <pmcmanus@mozilla.com <mailto:pmcmanus@mozilla.com>> wrote:
>>
>>             On Wed, 2012-04-04 at 07:02 +0000, Poul-Henning Kamp wrote:
>>             > In message <20120404054903.GA13883@1wt.eu
>>             <mailto:20120404054903.GA13883@1wt.eu>>, Willy Tarreau
>>             writes:
>>             >
>>             > >> I'm starting to get data back, but not in a state
>>             that I'd reliably
>>             > >> release. That said, there are very clear indicators
>>             of intermediaries
>>             > >> causing problems, especially when the pipeline depth
>>             exceeds 3 requests.
>>             >
>>             > I always thought that the problem in HTTP/1.x is that
>>             you can never
>>             > quite be sure if there is an un-warranted entity
>>             comming after at GET,
>>
>>             its not uncommon to have the consumer RST the whole TCP
>>             session when
>>             asked to recv too far beyond the current request it is
>>             processing. For
>>             some devices "too far" appears to be defined as "any new
>>             packet". I
>>             presume some variation of this is where Will's data point
>>             comes from.
>>             (Often 3 uncompressed requests fit in 1 packet).
>>
>>             That class of bug sounds absurd, but its really a pretty
>>             common pattern.
>>             As an example: hosts that fail TLS False Start (for which
>>             I understand
>>             second hand that Chrome needs to keep a black-list),
>>             react badly because
>>             there is TCP data queued when they are in a state that
>>             the expect their
>>             peer to be quiet. Same pattern.
>>
>>             The lesson to me is that you want to define a tight set
>>             of functionality
>>             that is reasonably testable up front - and that's what
>>             you can depend
>>             widely on later. Using anything beyond that demands
>>             excessive levels of
>>             pain, complexity, and cleverness.
>>
>>             (and all this pipelining talk as if it were equivalent to
>>             spdy mux is
>>             kind of silly. Pipelining's intrinsic HOL problems are at
>>             least as bad
>>             of an issue as the interop bugs.)
>>
>>             -Patrick
>>
Received on Thursday, 5 April 2012 18:19:01 GMT

This archive was generated by hypermail 2.2.0+W3C-0.50 : Friday, 27 April 2012 06:51:59 GMT