libwww webbot

Hello,

If you are interested, I had to add an additional include to HTRobot.c
so it would compile.  I kept getting
compile errors:

<snip>
.../../modules/md5/.libs/libmd5.so -ldl -Wl,--rpath -Wl,/usr/local/lib
HTRobot.o: In function `calculate_linkRelations':
/usr/local/src/w3org/libwww/w3c-libwww-5.4.0_gafilterfish/Robot/src/HTRobot.c:146:
undefined reference to `HTSQLLog_addLinkRelationship'
/usr/local/src/w3org/libwww/w3c-libwww-5.4.0_gafilterfish/Robot/src/HTRobot.c:179:
undefined reference to `HTSQLLog_addLinkRelationship'
/usr/local/src/w3org/libwww/w3c-libwww-5.4.0_gafilterfish/Robot/src/HTRobot.c:207:
undefined reference to `HTSQLLog_addLinkRelationship'
HTRobot.o: In function `Robot_delete':
/usr/local/src/w3org/libwww/w3c-libwww-5.4.0_gafilterfish/Robot/src/HTRobot.c:808:
undefined reference to `HTSQLLog_close'
HTRobot.o: In function `redirection_handler':
/usr/local/src/w3org/libwww/w3c-libwww-5.4.0_gafilterfish/Robot/src/HTRobot.c:1011:
undefined reference to `HTSQLLog_addLinkRelationship'
HTRobot.o: In function `terminate_handler':
/usr/local/src/w3org/libwww/w3c-libwww-5.4.0_gafilterfish/Robot/src/HTRobot.c:1089:
undefined reference to `HTSQLLog_addEntry'
HTRobot.o: In function `RHText_foundAnchor':
/usr/local/src/w3org/libwww/w3c-libwww-5.4.0_gafilterfish/Robot/src/HTRobot.c:1334:
undefined reference to `HTSQLLog_addLinkRelationship'
/usr/local/src/w3org/libwww/w3c-libwww-5.4.0_gafilterfish/Robot/src/HTRobot.c:1409:
undefined reference to `HTSQLLog_addLinkRelationship'
HTRobot.o: In function `RHText_foundImage':
/usr/local/src/w3org/libwww/w3c-libwww-5.4.0_gafilterfish/Robot/src/HTRobot.c:1445:
undefined reference to `HTSQLLog_addLinkRelationship'
/usr/local/src/w3org/libwww/w3c-libwww-5.4.0_gafilterfish/Robot/src/HTRobot.c:1495:
undefined reference to `HTSQLLog_addLinkRelationship'
RobotMain.o: In function `main':
/usr/local/src/w3org/libwww/w3c-libwww-5.4.0_gafilterfish/Robot/src/RobotMain.c:454:
undefined reference to `HTSQLLog_open'
/usr/local/src/w3org/libwww/w3c-libwww-5.4.0_gafilterfish/Robot/src/RobotMain.c:460:
undefined reference to `HTSQLLog_makeRelativeTo'
collect2: ld returned 1 exit status
make[4]: *** [webbot] Error 1
make[4]: Leaving directory
`/usr/local/src/w3org/libwww/w3c-libwww-5.4.0_gafilterfish/Robot/src'
make[3]: *** [all-recursive] Error 1
make[3]: Leaving directory
`/usr/local/src/w3org/libwww/w3c-libwww-5.4.0_gafilterfish/Robot/src'
make[2]: *** [all-recursive] Error 1
make[2]: Leaving directory
`/usr/local/src/w3org/libwww/w3c-libwww-5.4.0_gafilterfish/Robot'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory
`/usr/local/src/w3org/libwww/w3c-libwww-5.4.0_gafilterfish'
make: *** [all-recursive-am] Error 2


so I added HTSQLLog.h to HTRobot.c, i.e.

<snip>
#include "HTRobMan.h"
#include "HTQueue.h"
#include "HTAncMan.h"
#include "../../Library/src/HTSQLLog.h"
<snip>

and then reconfigured and did a 'make all' and libwww compiled and made
the webbot.  Now all I have to do is figure out how it works :)

I have used libww libraries in other projects, but never had a need for
the webbot, now I do!

I have a special project that I am working on with Squid, Glimpse,
WebGlimpse, Harvest, and libwww, so I need to understand how spiders
and/or webbots work, and how I can get them to feed data into a Squid
caching server.

Any ideas would be very helpful,

Murrah Boswell
otrcomm@isp-systems.com

Received on Thursday, 5 February 2004 05:50:18 UTC