Broken pipes & lost requests

Hi all, 
I am working on a robot-like client application using libwww to submit big amounts of GET requests to the server side.
Time ago we came across a memory growth problem due to the HTAnchor structures lifetime (once allocated they are NEVER freed till the end of the program). Trying to solve this problem with a periodic "garbage collection" of old 
HtAnchor objects every N submitted requests, we noticed that the library counter of active HTNet objects "HTNetCount" could remain indefinitely postive as if some requests had got lost.
Consider that the HTNetCount is incremented every time an HTNet object is added to the NetTable (Hash table containing all the HTNet) and decremented when, after the reception of the response to the submitted request, the matching HTNet object is deleted and removed from the NetTable.

We noticed that this requests loss took place only if there had previously been some SIGPIPE signal reception while
writing to  the network (HTWriter_write() in HTWriter.c).
That led us to suspect the presence of a BUG as in the HTTP state machine realized in the HTTPEvent() function ( HTTP.c module) the reception of a SIGPIPE after a write does NOT cause a recovery. In fact in case of broken pipe the returned value HT_CLOSED is never checked. I suggest a behaviour similar to that after a HTHost_read (l.1249 in HTTP.c, HTTPEvent() function): in case of HT_ERROR kill the pipeline, in case of broken pipe (HT_CLOSED return value) try to recover the pipeline.

This is the bug-fix I'm proposing (HTTP.c line 1236):

/* Now check the status code */
if (status == HT_WOULD_BLOCK)
    return HT_OK;
else if ( status == HT_PAUSE | | status == HT_LOADED) {
    type = HTEvent_READ;
} else if ( status == HT_ERROR) {
    http->state = HTTP_KILL_PIPE;
} else if ( status == HT_CLOSED )
    http->state = RECOVER_PIPE;

instead of:
 
/* Now check the status code */
if (status == HT_WOULD_BLOCK)
    return HT_OK;
else if ( status == HT_PAUSE | | status == HT_LOADED) {
    type = HTEvent_READ;
} else if ( status == HT_ERROR)
    http->state = HTTP_RECOVER_PIPE;


 And now 2 questions to the libww community:
1) Is what I have noticed really a bug or is there any reason not to recover after a sigpipe in the write branch ?
 
2) The buffer toward the network is flushed also when an HTTimer bound to an HTNet object is dispatched. In that case
    within the FlushEvent() in HTBufWrt.c  the return  value from the HTBufferWriterFlush() is NOT CHECKED AT ALL !     
    Even in this case after a SIGPIPE the pipelined HTnet objects flushed away will never be recovered.
    We tried forcing an HTHost_recoverPipe() when HT_CLOSED is returned but this attempt of recovery unfortunely 
    causes seg.violations (!!!!!) and it's not appliable . 
    Is there some other way to recover the pipe in this case? Don't you think this this FlushEvent() function needs 
    to be revised anyway (some sort of ret. value check)? 

HELP! Has anybody encountered similar problems? 
Thanks ! 
                   Azzurra


Azzurra Pantella
Mobile Applications Area
NETikos S.p.A.
Via Matteucci 34/B
56124 PISA
Tel.: + 39 050 968526
Fax: + 39 050 968525
e-mail:  azzurra.pantella@netikos.com
Internet: www.netikos.com