W3C home > Mailing lists > Public > ietf-http-wg-old@w3.org > September to December 1999

Re: Server-side roles in the HTTP

From: <kugler@us.ibm.com>
Date: Thu, 2 Sep 1999 09:51:40 -0600
To: jpierre@netscape.com (Julien Pierre)
cc: http-wg@cuckoo.hpl.hp.com, http-wg@hplb.hpl.hp.com
Message-ID: <872567E0.00575D31.00@d53mta08h.boulder.ibm.com>

jpierre wrote:
>kugler@us.ibm.com wrote:
>> > Servers MUST support a persistent connection if the content
>> >   generator supplies a Content-Length header. If it is not available,
>> >  the server SHOULD attempt to buffer the response in order to
>> >   generate one, although this MAY be circumvented if:
>> >   * the server does not have resources (i.e., memory) to do so, or
>> >   * the object is very large, and overall latency becomes
>> >   unacceptable, or
>> >   * the time required to generate the object adds unacceptable latency
>> ...
>> >  Servers MUST serve chunked encoding responses for all objects, if:
>> >   * Content-Length is unavailable and impractical to generate
>> >   and
>> >   * the client advertises itself as HTTP/1.1 capable, or
>> >   * the client includes 'chunked' in a TE header
>> Why wouldn't streaming the response with the "chunked" encoding be preferable
>> buffering and generating a Content-Length?
>It depends on the size of the object and how long it takes to generate the
>If the content is small and generated quickly, it makes sense for the server to
>buffer the response and transparently add a content-length header, as opposed
>using chunking. Chunking for small objects will waste bandwidth unnecessarily
>provide no benefits if the full content is immediately available.

If the full content is immediately available, it would be sent as one chunk.
How does that waste bandwith?   Let's see:  "Content-Length: xxxx" header, about
20 bytes; vs. "Transfer-Encoding:  chunked" header, about 27 bytes.  A
chunk-length and CRLF, say 6 bytes, and a 0,CRLF,CRLF terminator, say 5 more
bytes.  So we're looking at about 38 bytes of overhead vs. 20.   Is that

Chunking does provide benefits if the full content is not immediately available,
and adds (what I consider to be) an insignificant cost if it is not, so I think
it would be better to default to chunking instead of using a bunch of heuristics
to try to coalesce response content.

Either that, or provide a way for "content-producers" to control the

>If the application generates too much content to buffer, or ends up taking too
>to generate little content, then the server can choose to chunk.
>But this may be a matter for webmasters do decide depending on their
>how much bandwidth they have, and how much latency they can accept, so we are
>those time and buffer size thresholds configurable. We may also add settings
for the
>minimum size of the chunks themselves to conserve bandwidth.
>> > 6.4 Transfer-Encoding
>> >   Servers SHOULD reply to requests that allow transfer encoding of
>> >  objects (i.e., TE header present) with appropriate encoding, in a
>> >  fashion transparent to the content generator.
>> This paragraphs seems to suggest that transfer encoding is only allowed if
>> TE request-header is present.  However, if the TE field-value is empty or if
>> TE field is present, the transfer-coding "chunked" is allowed, and is always
>> acceptable.
>Good point - looks like Mark should add a provision for this case.
>> >   There MUST be a method by which content generators can specify that
>> >   content is not to be buffered; this MAY be performed by a pseudo-
>> >   HTTP header that is consumed by the server.
>> I think you're only addressing generated (response) content in this
>> I'd also like to have a method by which content generators can specify that
>> received (request) content is not to be buffered.   Sometimes the "content
>> generator" can be doing something useful with the content of a large POST
>> request while it's still being received (or even while it's still being
>> generated by the client).  In some applications (e.g., IPP), POST content may
>> so large that you don't even want the server to attempt to buffer it.
>This won't work for some content generators.
>For example, you can't start a CGI before you know the length of the POST data
>because you have to set it in the CGI process environment variables.

The one pathological case is CGI/1.1 and chunked POST requests.  Supposedly,
future versions of CGI will deal with chunking.  And if the request provides
Content-Length, you don't have to buffer it, even for CGI/1.1.

>for a good time, try kill -9 -1
> - jpierre.vcf
> - smime.p7s
Received on Thursday, 2 September 1999 16:55:48 UTC

This archive was generated by hypermail 2.4.0 : Friday, 17 January 2020 17:16:34 UTC