Re: HTLoadToFile crashing after load on FTP URLs, HTRequest_bytesRead inoperant

On Mon, May 05, 2003 at 05:10:02PM +0200, Timothee Besset wrote:
> I keep calling HTRequest_bytesRead on each HT_PROG_READ alert callback,
> but it remains 0. I can see the file being downloaded and growing though.

Hmm, my code uses HTResponse_length(HTRequest_response(request)). I'm not 
sure whether that works though, since I'm keeping track of the length 
myself regardless of what libwww says about it.

> - After the download has completed, it crashes. Here's what the end of the
> trace says:
[...]

It works for me! See the end of this mail for the debug output I get (with
the OpenBSD ftpd and my respective FTP patch applied).  Your FTP server
seems to close the connection on you, mine doesn't. Maybe that makes a
difference.

[This FTP patch of mine is unlikely to affect your problem, it only
influences libwww's behaviour before the FTP GET. Check the list archives
if you need it.]

> HTDoClose... Close 20
> Net Manager. Decreasing active sockets to 0, 0 persistent sockets
> FTP Get Data now in state NEED_BODY

Are you maybe making "recursive" calls to libwww? libwww doesn't like it at
all e.g. if you start/stop requests from within callbacks which you
registered with it.

> Actually, I know how the crash happens, but I haven't thought of a
> solution. I thought for a while that it could be related to the ftp
> daemon, but I tried a win32 daemon and proftp, and they both cause the
> same crash.

If you want to have an envious look at Code That Works (woohoo!), look at
the things in src/net inside <http://atterer.net/jigdo/jigdo-0.7.0.tar.bz2>
;-)

[...]
> status is HT_CLOSED
> ctrl->repcode is 150
> data->complete is 1

FTP code 150 is "File status okay; about to open data connection", looks
like this is still the last status code *before* the error. 

> But the data->complete |= 2 doesn't trigger. If it would, I *think* that
> HTFTP.c would finish the job and exit cleanly.

I'm not sure what the problem is - please find out whether the server is
using active FTP, whether using passive FTP makes a difference, and whether
the socket that's unexpectedly being closed by the remote end is for the
control or the data connection.

Cheers,

  Richard

-- 
  __   _
  |_) /|  Richard Atterer     |  CS student at the Technische  |  GnuPG key:
  | \/¯|  http://atterer.net  |  Universität München, Germany  |  0x888354F7
  ¯ '` ¯
...
Read Socket. 2669 bytes read from socket 7
Host Event.. READ passed to `ftp://localhost/image'
FTP Event... now in state FTP_NEED_DATA
FTP Get Data now in state NEED_BODY
Read Socket. FIN received on socket 7
HTDoClose... Close 7
Net Manager. Decreasing active sockets to 1, 1 persistent sockets
FTP Get Data now in state SUB_SUCCESS
FTP Event... now in state FTP_SUCCESS
Channel..... Delete input stream 0x8247078 from channel 0x82248f0
Channel..... Delete input stream 0x8247078 from channel 0x82248f0
Net Object.. Delete 0x8225a70 and call AFTER filters
Host info... Remove 0x8225a70 from pipe
Host Object. closing socket -1
Channel..... Semaphore set to 0 for channel 0x82248f0
Channel..... Delete 0x82248f0 with semaphore 0, status 900
Channel..... Semaphore decreased to 0 for channel 0x82248f0
Host info... removed host 0x8237f98 as persistent
Net Object.. Check for pending Net objects
Net Object.. Freeing object 0x8225a70
Net Object.. Delete 0x823ccc8 and call AFTER filters
Host info... Remove 0x823ccc8 from pipe
Host Object. keeping persistent socket 5
Channel..... Delete 0x822c598 with semaphore 1, status 200
Channel..... Delete input stream 0x823f058 from channel 0x822c598
Channel..... Delete input stream 0x823f058 from channel 0x822c598
Channel..... Semaphore decreased to 0 for channel 0x822c598
Timer....... Created one shot timer 0x822cb10 with callback 0x809e9e8, 
context 0x8224940, and relative timeout 60000
Host........ Object 0x8224940 going idle...
Net Object.. Check for pending Net objects
Net Object.. Freeing object 0x823ccc8
Net After... calling 0x809363c (request 0x8224a60, response 0x823a4c8, 
status 200, context (nil))

Received on Monday, 5 May 2003 14:15:42 UTC