European WWW indexing coordination

[Apologies for cross-posting :-)]

TF-CHIC:
TERENA Task Force on Cooperative Hierarchical Indexing Coordination

When a user wishes to find specific information on the World-Wide Web
today a common approach is to use one of the established commercial
search engines such as AltaVista, Lycos, or WebCrawler.  The majority
of these services make inefficient use of international bandwidth by
running their data gathering robots across the commodity Internet from
central sites (typically in the United States), and by requiring end
users to connect to these central sites in order to perform searches.
Whilst World-Wide Web indexing and searching would appear to be major
consumers of international bandwidth, little hard information on the
cost of these activities is available.

In addition to these commercial services, a number of local, regional
and topical WWW indexes have been created, as grass-roots volunteer
projects or with public funding.  These are of particular interest to
the academic and research community because they can provide a higher
signal to noise ratio in search results, and in several cases employ
advanced techniques such as distributed data gathering and searching
so as to make more effective use of network bandwidth.

In order that the current situation and possibilities for future
development be better understood, a TERENA task force has been formed.
This will initially be:

  (a) gathering statistics on the usage of WWW search engines from
      proxy cache server and network managers.

  (b) gathering statistics on robot activity from HTTP server
      administrators.

  (c) identifying existing grass-roots WWW indexing and searching
      initiatives, and the potential for coordination.

  (d) identifying technologies currently in use for distributed
      indexing and searching, and emerging technologies.

Whilst this effort is primarily aimed at members of the European
academic and research community, any constructive participation would
be very welcome.  Please feel free to forward this message on to anyone
who might be interested.

You can join the task force's mailing list by sending email to:

        mailserver@terena.nl

containing only the text:

        subscribe tf-chic My Name

replacing "My Name" as appropriate.

As the task force progresses, we will be putting our findings up on the 
TERENA WWW server, at:

  <URL:http://www.terena.nl/task-forces/tf-chic/>

Our first face to face meeting will be just before the Joint European
Networking Conference, on the morning of May 11th in Edinburgh, UK.

Cheers!

Martin Hamilton <martin@mrrl.lut.ac.uk>
Sigfrid Lundberg <siglun@gungner.ub2.lu.se>
-TF-CHIC co-chairs