- From: Ville Skyttä <ville.skytta@iki.fi>
- Date: Sun, 15 Feb 2004 13:16:43 +0200
- To: Nick Kew <nick@webthing.com>
- Cc: Semaphore Corporation <help@semaphorecorp.com>, www-validator@w3.org
On Sun, 2004-02-15 at 12:19, Nick Kew wrote: > On Sat, 14 Feb 2004, Semaphore Corporation wrote: > > > > > needs a delaytime-between-links option so it doesn't thrash the server. > > Yep. That's the difference between a well-behaved spider and a > while-you-wait online service. The two are not compatible. On the other hand, when run as a CGI (ie. the online version), it already sleeps 3 seconds between checked documents (note: documents, not links). I'm not sure how useful that feature is though. Maybe it was meant to wait between checking both links and documents, and is just buggy. Comments welcome...
Received on Sunday, 15 February 2004 06:16:45 UTC