FTP client help

I'm looking to use libwww solely for the FTP client capabilites.  I've been
all through the sample apps and read just about every FTP-related post in
the mail list archive but am still experiencing strange behaviour in even
the smallest of my test apps.
I believe the following to be correct code to retrieve a directory listing
and then a binary file over FTP:

void ftptest(char *dir_url, char *file_url)
{
    HTRequest *request;
    HTChunk *chunk;

    HTProfile_newNoCacheClient("ftptest", "0.1");
    request = HTRequest_new();

    /* get directory listing */
    HTRequest_setOutputFormat(request, WWW_RAW);
    HTRequest_addConnection(request, "close", "");
    HTFTP_setTransferMode(FTP_DIR_TRANSFER_MODE);

    chunk = HTLoadToChunk(dir_url, request);
    if ((chunk != NULL) && (HTChunk_size(chunk) > 0)
    {
        char *dir = HTChunk_data(chunk);
        printf(dir); 
    }
    HTChunk_delete(chunk);
    HTRequest_clear(request);

    /* get file */
    HTRequest_setOutputFormat(request, WWW_RAW);
    HTRequest_addConnection(request, "close", "");
    HTFTP_setTransferMode(FTP_BINARY_TRANSFER_MODE);

    chunk = HTLoadToChunk(file_url, request);
    if ((chunk != NULL) && (HTChunk_size(chunk) > 0))
    {
        printf("Success!\n");
    }

    HTChunk_delete(chunk);
    HTRequest_delete(request);
    HTProfile_delete();
}

This will (usually!) return only the head of the directory listing, and then
a chunk with zero size for the file.

I have a few (probably obvious) questions and misunderstandings:

--- I understand the profile must not be pre-emptive for FTP, and so I have
used the NoCacheClient profile (?).  (I have managed to retrieve the full
directory listing and the binary file by using Preemptive. But it would
block occasionally).

--- Setting the output format to WWW_FTP_LNST or WWW_FTP_LIST simply returns
a zero size chunk for the directory listing. I presume I should be adding my
own Converter from text/html to text/x-ftp-list ?

--- Should the I handle the directory listing returned to me by using
HText_registerTextCallback() rather than the data returned in the chunk?
(the data in the chunk is in HTML, even if WWW_PLAINTEXT is specified and
HTFormat_addConversion("text/html", "text/plain", HTMLToPlain...) is
registered - I thought setting the output format to WWW_RAW should leave the
directory listing as it was received rather than converting to HTML).

--- My "sop" tracing for the directory list retrieval ends up as follows:
    ...
    Reading...
        Host........ passing 55 bytes as consumed to c90b8
        Host........ 0 bytes remaining 
        FTP Rx...... `150 Opening ASCII mode data connection for file list.'
        Read Socket. Target returns 200
        FTP Get Data now in state NEED_ACCEPT
        Accepted.... socket 6
        Host connect Unlocking Host c71b8
        FTP Get Data Passive data socket 6
        FTP Get Data now in state NEED_STREAM
        StreamStack. Raw output...
        Error....... Add   2    Severity: 8     Parameter: `Unspecified'
Where: `HTDir_new'
        HTDir_new... base is `test/'
        FTP Get Data now in state NEED_BODY
        Read Socket. WOULD BLOCK fd 6
...and then exits out with (usually) a 96-byte chunk containing just the
directory header.

--- My "sop" tracing for the file retrieval ends up as follows:
        ChunkStream. Chunk c45b0 created with max size 0
        HTAccess.... Accessing document ftp://foo:bar@example.com/test.bin
        Net Before.. calling 6c6fc (request c4b18, context 0)
        Net Before.. calling 6c574 (request c4b18, context 0)
        Net Object.. c78c0 created with hash 3
        Net Object.. starting request c4b18 (retry=2) with net object c78c0
        FTP......... Looking for `ftp://foo:bar@example.com/test.bin'
        Net Object.. c7a00 created with hash 4
        Net Object.. Duplicated c78c0
        FTP Event... now in state FTP_BEGIN
        FTPParse.... uid `foo' pw `bar'
        FTPParse.... Datatype ?
        FTP Event... Transfer mode set to 'I'
        FTP Event... now in state FTP_NEED_CONN
        HTHost parse Looking up `example.com' on port 21
        Host info... REUSING CHANNEL c7300
        Host info... Added Net c78c0 (request c4b18) as pending on Host
c66c8, 1 requests made, 1 requests in pipe, 1 pending
        HTDoConnect. Pending...
...and then exits out with a zero size chunk (even when not re-using a
'request'). 

I've tried applying the FTP patch submitted to the list a couple of months
ago by Peter Stamfest, but it did not help my situation (which leads me to
believe I'm misunderstood something quite trivial - here's hoping).

By adding some simple retries in FTPEvent() upon receiving a WOULDBLOCK I
get all my data just fine.  I realise this is completely the wrong thing to
do and suspect there is something I should be registering beforehand?

(I'm using libwww 5.3.1, on Solaris 2.6 & 7)

Thanks very much for any advice or insight,
Ken.

Received on Tuesday, 31 October 2000 03:30:43 UTC