- From: Matthew Carey <matthew@ssl.co.uk>
- Date: Tue, 02 Mar 2004 17:08:39 +0000
- To: www-talk@w3.org
The webbot goes into an infinite tight loop on some sites. (latest checkout) When started like this: ./webbot -depth 99 -v -prefix 'http://www.uni4me.com/uni4me' 'http://www.uni4me.com/uni4me/index.cfm' Most of the time it gets stuck a little way into walking this site. Now this may be because some vital parameter is missing but it works with most sites just the occaisonal site causes problems. This happens right down in the event loop. Any ideas? Host Event.. WRITE passed to `http://www.uni4me.com/uni4me/what-is-uni/index.cfm#' HTDoConnect. WOULD BLOCK `www.uni4me.com' Memory Free. 0x80e5d10 Event Loop.. calling select: maxfds is 4 ............ READ : ............ WRITE: 4 ............ OOB : ............ Timeout is 20 s, 0 microsecs Event Loop.. select returns 1 ............ READ : ............ WRITE: 4 ............ OOB : ............ Timeout is 20 s, 0 microsecs Memory Free. 0x80ac700 Timer....... Found timer 0x80dff78 with callback 0x8050cec, context 0x80c7e48, and relative timeout 20000 EventOrder.. execute ordered events Memory Free. 0x80ac9b0 EventList... calling socket 4, request 0x80c4ef0 handler 0x8079608 type HTEvent_WRITE Host Event.. WRITE passed to `http://www.uni4me.com/uni4me/what-is-uni/index.cfm#' HTDoConnect. WOULD BLOCK `www.uni4me.com' Memory Free. 0x80ab740 Event Loop.. calling select: maxfds is 4 ............ READ : ............ WRITE: 4 ............ OOB : ............ Timeout is 20 s, 0 microsecs Event Loop.. select returns 1 ............ READ : ............ WRITE: 4 ............ OOB : ............ Timeout is 20 s, 0 microsecs Memory Free. 0x80adae8 Timer....... Found timer 0x80dff78 with callback 0x8050cec, context 0x80c7e48, and relative timeout 20000 EventOrder.. execute ordered events Memory Free. 0x80c51e0 EventList... calling socket 4, request 0x80c4ef0 handler 0x8079608 type HTEvent_WRITE Host Event.. WRITE passed to `http://www.uni4me.com/uni4me/what-is-uni/index.cfm#' HTDoConnect. WOULD BLOCK `www.uni4me.com'
Received on Tuesday, 2 March 2004 12:12:25 UTC