W3C home > Mailing lists > Public > www-lib@w3.org > October to December 1999

How to map out an entire Web site

From: Q. Alex Zhao <azhao@cc.gatech.edu>
Date: Thu, 16 Dec 1999 14:49:14 -0500 (EST)
Message-Id: <199912161949.OAA00641@gvu2.cc.gatech.edu>
To: www-lib@w3.org
I want to get a list of URLs to all the images on a Web site and I'm having
problems using "webbot".

I tried
	webbot -depth 10 -prefix $url -img -imgprefix $url -hit hit.log $url
and lots of other variations of the command on a RedHat Linux 6.1 machine
(url was "http://www.cc.gatech.edu/gvu/ii/", program version was 5.2.8),
but the log file didn't contain the URLs to all the images.

"webbot" didn't traverse into the sub-directories such as
"http://www.cc.gatech.edu/gvu/ii/community/" -- it said something like
"does not fullfill constraints". What are the constraints and why it didn't
satisfy the constraints?

= Q. Alex Zhao
  mailto:azhao@cc.gatech.edu voiceto:404-894-9390 faxto:404-385-1253
  Graphics, Visualization & Usability Center, Georgia Inst. of Tech.
Received on Thursday, 16 December 1999 14:49:17 UTC

This archive was generated by hypermail 2.3.1 : Tuesday, 6 January 2015 21:33:52 UTC