Re: Padding size in chunked encoding

On Fri, 21 Feb 1997, Henrik Frystyk Nielsen wrote:

> At 08:05 AM 2/21/97 -0600, John Franks wrote:
> 
> >The pipelining paper emphasizes the importance of avoiding small TCP
> >packets.  This seems awkward to do efficiently with the chunked
> >encoding unless it is somehow possible to insert the size of a chunk
> >at the beginning of a buffer after the data has been put in the
> >buffer.  The alternative of moving all the data in the buffer to
> >adjust the amount of space at the beginning is not appealing.  And, of
> >course, the main point of chunking is to deal with situations where we
> >don't know the size in advance.
> 
> In the particular case of the libwww pipelining implementation, buffering
> is not done at the HTTP client level but at a lower level:
> 
>             Request 1        Request 2         Request 3
> 
>                            Buffer Stream
> 
>                               Network
> 
> That is, each request just writes and the buffer stream is responsible for
> flushing the data as a function of time and buffer size. That way, you can
> get multiple requests (or responses) to fit into same TCP segment if their
> sizes allow it.
> 

>From an efficiency point of view this is the same as moving all the
data to make room for the chunk size.  I.e.  (if I understand
correctly) you buffer a chunk for request 1 at the application level
then write its size to the buffer stream then copy the chunk to the
buffer stream.  The data in the chunk gets buffered twice.  What would
be nice would be if the buffer stream could take care of the chunking
and the application layer could just communicate the start of
chunking, the data, and the end of chunking.  But I don't see how to
do that without buffering every chunk twice or using a "pad"
extension.

> However, even if you are using "direct" buffering then I can't see why you
> can't count backwards from the start of the data and then let the start of
> the size part vary (no param in this example):
> 
>                        x x x x x x | x x x x x x ...
>                                  |   |
>                      <- size start   data start ->
> 
> or use vwrite for writing multiple buffers at once.
> 

This would work for one chunk at the beginning of a buffer but if
multiple requests get sent to the buffer stream it can't do
appropriate chunking without moving data around or some way to
pad the chunk size.

John Franks 	Dept of Math. Northwestern University
		john@math.nwu.edu

Received on Friday, 21 February 1997 10:41:06 UTC