- From: Martijn Koster <mak@surfski.webcrawler.com>
- Date: Mon, 11 Dec 1995 19:26:55 -0800
- To: www-talk@w3.org
Steve <steve@mordred.gatech.edu> wrote: >I may have missed it in the archives, but I didn't see any topics on which > sections of a document a robot or spider or other Web traveller might > examine to determine content information. "Might": every part. In practice, who knows, they all do different things. >If there isn't a specific indexing method that is standard for all search >engines/robots/crawlers, shouldn't there be one? What, and give up competition? :-) But yes, it'd help to have better ways. Frank Atkinson <FRANK@han1.hannah.com> replied: > There is a mailing list at autoshoppers@han1.hannah.com (put subscribe in the > subject to subscribe) examining some of these issues as they relate to bots > which will go "shopping" on the web. Don't know if that has a different focus, but there is the robots mailing list on robots-request@webcrawler.com -- Martijn __________ Email: m.koster@webcrawler.com WWW: http://info.webcrawler.com/mak/mak.html
Received on Monday, 11 December 1995 22:27:01 UTC