Re: Suggestion for NEW Issue: Pipelining problems

I'm interested in this, purely to try and write a "standards compliant" pipeline
support for a future Squid version rather than hacking up something local
with (lots of) heuristics.

I'd really be interested in a pipeline negotiation procedure to tag a peer
or connection as correctly implementing 'standards' pipelining.

If thats not possible (and its certainly outside the scope of your post Mark)
then I'd again be interested in a collaborative document outlining how
pipelining is actually currently implemented in various servers/intermediaries/UAs
rather than each trying their own implementation.

(That, and I'm interested in SCTP too..)



Adrian

On Thu, Feb 28, 2008, Mark Nottingham wrote:
> 
> There hasn't been any traffic on this thread for some time, and I  
> neglected to open an issue for it. See: 
> <http://www3.tools.ietf.org/wg/httpbis/trac/ticket/99 >.
> 
> Based on the discussion so far, I think this is where we are WRT  
> pipelining:
> 
> a) In the scope of HTTPBIS, the most we can do is insert some  
> implementation advice that pipelining may be incorrectly implemented  
> by origin servers and intermediaries (including "hidden" infrastructure)
> b) However, we shouldn't deprecate it, as it is still useful (and  
> actively used) in some cases
> c) "fixing" the pipelining problems may involve a new transport  
> protocol (see discussion of SCTP) or other mechanism, but is outside  
> the scope of HTTPBIS.
> 
> To that end, I propose we add implementation advice as per (a) and  
> close the issue. Thoughts?
> 
> If that sounds reasonable to people, I can trawl the archives for  
> different issues that might be encountered as fodder for  
> implementation advice.
> 
> Cheers,
> 
> 
> On 09/08/2007, at 5:20 AM, Justin Erenkrantz wrote:
> 
> >
> >On 7/23/07, Roy T. Fielding <fielding@gbiv.com> wrote:
> >>of the network.  Likewise, pipelining has huge benefits for
> >>specialized HTTP services, such as Subversion (when it uses it)
> >>and content management systems.
> >
> >ra_serf (optional in Subversion 1.4+ and using the serf client
> >library) uses pipelining.
> >
> >The pipelining problems I've seen in the real-world are:
> >
> >- Not knowing how many requests this connection will be able to serve.
> >The default in httpd is 100 requests per connection, but
> >svn.collab.net tuned it down to 10 - which caused all sort of fun when
> >testing ra_serf against Subversion's own repository.  I eventually got
> >them to up it back to the default.  serf tries to figure out the limit
> >heuristically (i.e. write as much as possible, then count how many
> >responses we got before the connection closed - that'll be the limit
> >going forward).
> >
> >- Lost responses are, sadly, real.  Later versions of httpd 2.0.x
> >re-introduced a lingering close bug where it won't wait for the last
> >packet to be written; this is fixed in 2.2.x, but have to be accounted
> >for.
> >
> >In general, what I settled upon is that serf remembers all in-flight
> >requests so that if we don't get a response back we can re-send them
> >if something happened (i.e. lost response or hitting the cap on the
> >number of requests on that connection).  It's really the only reliable
> >mechanism for dealing with pipelined requests.
> >
> >Out of order pipelining would be nice for things like WebDAV, but a
> >bigger problem is that the WebDAV protocol requires specific methods
> >to be executed in lock-step using the results from the prior response
> >as input for the next request.  This is the largest protocol headache
> >I have with WebDAV at the moment.  Subversion would ideally like to be
> >able to use pipelining commits with WebDAV, but that'll likely require
> >us forking the protocol in some sense to let the underlying methods be
> >executed out-of-order as well.  -- justin
> >
> 
> 
> --
> Mark Nottingham     http://www.mnot.net/
> 

-- 
- Xenion - http://www.xenion.com.au/ - VPS Hosting - Commercial Squid Support -
- $25/pm entry-level VPSes w/ capped bandwidth charges available in WA -

Received on Thursday, 28 February 2008 01:40:22 UTC