RE: Http overhead

It did was a while ago, when I did some experimenting (personal purpose)
with HTTP. Your server (MSN Search) didn't return a content-length field
or send the data chunked (this is a good while ago)... Also another
server from CNET did the same thing. At least this was about a year ago,
or maybe even a longer time a ago, this wasn't recently....

I don't know how your search servers work, but maybe the server just
forwarded a response from another server. The URL I gave to the server
(using telnet) I had from the search area in IE 4. So maybe actual data
came from someone like Infoseek or Aslavista....
The server however did turn, at that time, "Server: Microsoft-IIS[...]".

At that time I was developing a simple proxy server (for myself), that
ran into trouble with sites not giving a content-length or using chunked
tranfer-encoding. The first trouble I then developed support was chunked
tranfer encoding, what did turn out to work correct. But at this time I
still couldn't download from MSN search of CNET. Possiblity that it was
this error.

Also the proxy didn't support caching (yet) and did turn out to be quite
unreliable, especially with subsequent requests on a connection.


But don't forget I is a long while ago, what I actually should have put
in the previous mail (my fault). Software and systems evolve, and you
probably don't have the same stuff you had a couple years ago...


- Joris


> -----Original Message-----
> From: Chris Wendt [mailto:christw@microsoft.com]
> Sent: Saturday 02 December, 2000 5:12
> To: 'joris.dobbelsteen@mail.com'
> Subject: RE: Http overhead
> 
> 
> Joris, 
> 
> We in MSN Search do populate the content length field in the 
> header, just
> verified. Could you point out what you consider wrong in what you are
> seeing?
>  
> In general, with IIS, if the client is HTTP 1.0 (old), and 
> the server ASP
> dumps the buffer before the response ends, then there may be a chunked
> response.  I am not aware of our service ever invoking that 
> behavior.  This
> should be the same behavior for any web server. 
> 
> Chris.. 
>  
> -----Original Message----- 
> From: Joris Dobbelsteen [mailto:joris.dobbelsteen@mail.com] 
> Sent: Friday, December 01, 2000 1:24 PM 
> To: WWW WG (E-mail) 
> Subject: RE: Http overhead 
>  
> You are right, I did know this. 
> Indeed that's why I favor chunked encoding for dynamic 
> resources. MSN's 
> (search) server (MS-IIS) transfers the data by just closing the 
> connection at the end (not sending any indication about the 
> size of the 
> document) what also decreases the transfer 'reliability' - or better, 
> you don't know wether you have all or not... HTML or JPEG/GIF you can 
> guess it, but with many other sources you don't (like TXT).... 
> Also MSN's server has thus more overhead for the server and 
> waisted time 
> between the requests for the client... 
> - Joris 
> > -----Original Message----- 
> > From: francis@localhost.localdomain 
> > [mailto:francis@localhost.localdomain]On Behalf Of John Stracke 
> > Sent: Friday 01 December, 2000 16:04 
> > To: WWW WG (E-mail) 
> > Subject: Re: Http overhead 
> > 
> > 
> > Joris Dobbelsteen wrote: 
> > 
> > > Many apache servers send data chunked. Takes a couple bytes 
> > (average of 
> > > 4) for every block transfered. Maybe a total overhead of an 
> > additional 
> > > 20-40 bytes per transfer (maybe less). This is just a guess... 
> > 
> > Yes, but it's actually better than that: AFAIK, Apache uses chunked 
> > transfer-encoding only for dynamic resources, where it can't 
> > predict the 
> > content-length.  The alternative would be (a) buffer the 
> output before 
> > sending it down, or (b) defeat persistent connections.  
> > Either (a) or (b) 
> > would increase; (b) would actually cost extra bandwidth, 
> and (a) would 
> > cause bandwidth consumption to come in spikes.  So, most 
> > likely, the cost 
> > of chunking is lower than the cost of not chunking; it's 
> > certainly lower 
> > than the nominal overhead of the encoding. 
> > 
> > (Sorry to those to whom this is obvious--probably including 
> > Joris--but I 
> > didn't want to leave anybody thinking they could save 
> > bandwidth by turning 
> > off chunking.  :-) 
> > 
> > -- 
> > /=================================================================\ 
> > |John Stracke    | http://www.ecal.com |My opinions are my own.   | 
> > |Chief Scientist |================================================| 
> > |eCal Corp.      |In the country of the blind, the one-eyed man is| 
> > |francis@ecal.com|in therapy.                                     | 
> > \=================================================================/ 
> > 
> > 
> > 
> > 
>