wild socket problem with webbot

Hi,

I'm running webbot on linux and for some urls webbot seems to get stuck in
select loop in HTEvent_List_loop doing nothing but consuming a lot of CPU
cycles. This seems to happen when EventOrder_executeAndDelete calls 
HostEvent which immediatly returns HT_OK after following if statement

/* netscape and apache servers can do a lazy close well after usage of
 * previous socket has been dispensed by the library, the section below
 * makes sure the event does not get miss attributed
*/
if (HTChannel_socket(host->channel) != soc && type != HTEvent_ACCEPT &&
!host->listening) { 
    HTTRACE(CORE_TRACE, "Host Event..  wild socket %d type = %s real socket is %d\n" _ soc _
	type == HTEvent_CLOSE ?  "Event_Close" : "Event_Read" _
	HTChannel_socket(host->channel)); 
    return HT_OK; }

I'm afraid I don't understand what this if statement is supposed to be
doing. Below is the trace of what happens. I've deleted Memory Free lines
to make shorter. 

The first interesting thing seems to be when 

Event....... Couldn't find socket -1. Can't unregister type HTEvent_READ
Event....... Couldn't find socket -1. Can't unregister type HTEvent_WRITE

can anyone suggest a fix?


Net Object.. Persistent connection set ON succeeded
StreamStack. Constructing stream stack for message/rfc822 to www/debug
Error....... Add  39    Severity: 1     Parameter: `Not Supported'
Where: `HTTPNextState'
Host........ passing 28 bytes as consumed to 0x80da6c8
Host........ 981 bytes remaining 
MIME header. Server: Microsoft-IIS/4.0
Memory Free. 0x80ed120
MIME header. Date: Mon, 22 Nov 1999 13:55:51 GMT
MIME header. Connection: close
MIMEParser.. Close received...
MIME header. Content-Length: 850
MIME header. Content-Type: text/html
Alert Call.. No entry found for opcode 32
Host........ passing 131 bytes as consumed to 0x80da6c8
Host........ 850 bytes remaining 
Alert Call.. No entry found for opcode 32
Read Socket. Target returns 200
HTTP Clean.. Called with status -501, net 0x80f00b8
Net Object.. Delete 0x80f00b8 and call AFTER filters
Host info... Remove 0x80f00b8 from pipe
Host Object. got close notifiation on socket 4
25 requests made, 14 in pipe, max 11 requests pr connection
Channel..... Delete 0x80da688 with semaphore 14, status -501
Channel..... Delete input stream 0x80da6c8 from channel 0x80da688
MIME........ FREEING....
Channel..... Delete input stream 0x80da6c8 from channel 0x80da688
Channel..... Semaphore decreased to 13 for channel 0x80da688
Host recover 0x80da450 recovered 1 times. Moving 13 Net objects from pipe
line to pending queue
Event....... Couldn't find socket -1. Can't unregister type HTEvent_READ
Event....... Couldn't find socket -1. Can't unregister type HTEvent_WRITE
Host recover Resetting net object 0x80f3f50
HTTP Clean.. Called with status -904, net 0x80f3f50
HTTPGen..... ABORTING...
HTTPRequest. ABORTING...
Buffer...... ABORTING...
FileWriter.. ABORTING...

(above 6 lines repeated 12 more times with differnt addresses)


Net Object.. Check for pending Net objects
Net Object.. Freeing object 0x80f00b8
Net After... calling 0x804c7f4 (request 0x80eff18, response 0x80eb7e8,
status -501, context (nil))
Net After... calling 0x8056d28 (request 0x80eff18, response 0x80eb7e8,
status -501, context (nil))
Load End.... Request ended with code -501
Net After... calling 0x804c838 (request 0x80eff18, response 0x80eb7e8,
status -501, context (nil))
Host info... Looking for `209.249.196.102'
Host info... Found `209.249.196.102'
HTAnchor.... Clear all header information
Request..... Delete 0x80eff18
Response.... Delete 0x80eb7e8
Host Event.. wild socket 4 type = Event_Read real socket is -1
Event Loop.. calling select: maxfds is 4
............ READ : 4
............ WRITE:
............ OOB  :
............ Timeout is 19 s, 992000 microsecs
Event Loop.. select returns 1
............ READ : 4
............ WRITE:
............ OOB  :
............ Timeout is 20 s, 0 microsecs
Timer....... Found timer 0x80eaa78 with callback 0x805542c, context
0x80eaa30, and relative timeout 20000
EventOrder.. execute ordered events
EventList... calling socket 4, request 0x80f38c8 handler 0x8090738 type
HTEvent_READ
Host Event.. wild socket 4 type = Event_Read real socket is -1

and so on.........



-- 
----------------------------------------------------
 Tadhg O'Meara
 Dept. of Computer Science,
 University College Dublin,
 Belfield, Dublin 4, Ireland.
----------------------------------------------------

Received on Monday, 22 November 1999 09:08:44 UTC