Virtual newsgroups and creating the memory of Usenet via www and wais
Jyrki Kuoppala (jkp@cs.hut.fi)
Wed, 3 Jun 1992 18:57:19 +0300
Date: Wed, 3 Jun 1992 18:57:19 +0300
Message-Id: <199206031557.AA18097@lusmu.cs.hut.fi>
From: Jyrki Kuoppala <jkp@cs.hut.fi>
To: www-talk@nxoc01.cern.ch
Subject: Virtual newsgroups and creating the memory of Usenet via www and wais
Here's an article I sent a while ago to various Usenet newsgroups:
>With the enormous numbers of groups now existing, tracking
>through all of them to find interesting and relevant topics
>is becoming very hard, and it seems clear that the namespace
>is becoming hopelessly muddled by democracy in action, so
>a software solution might be the only way out.
An idea just popped up in my head.
Currently we have a news gateway to www (a www client can be used as a
newsreader), which helps to organized groups better than the flat
namespace of the newsgroups most other newsreaders seem to have. Ie. you
create www nodes which have topically similar newsgroups listed
together in a node and hypetext links to those newsgroups. Each user
can make personal nodes having the groups or use ones someone else has
published on the net.
Now, as www works very well with dynamic node creation (which is the
way the nntp and wais gateways work), to have a "virtual newsgroup"
you would just instead of having a link to a "real" newsgroup in a www
node have a link to a FIND node with suitable keywords, and then wais
would be used to search the articles mathing the find criteria and
would create a virtual node for the matches, like the nntp gateway
currently creates a virtual node with the current contents of a real
Usenet group.
As the www news client / gateway already has the capability to follow
links to earlier dicussions (References: field), following the tree
from the virtual newsgroups would be easy. For more usability, the
www / news gateway (or the news system) should also create forward
references, ie. to backtrack the References field when new articles
come in and thus keep track of articles which are comments to a
certain article.
To apply the same principles to other newsreaders, nntp should be
extended to have the "newsgroup" instead be a "search criteria", which
would be interpreted by the wais (or some other search engine) server
and transparent to nntpd (except if it's the old-fashioned newsgroup
name). People would have the search criteria - keywords from the
text, poster names, whatever - in their .newsrc files alongside with
the newsgroup names.
I'm not sure how to keep track of what articles the user has already
read, as there are no article numbers for the virtual groups. Keeping
track of message IDs takes a lot of space. But even as a browsing
tool it would be nice.
Hmm, come to think of it to combine a wais server indexing the news
database with the www / nntp gateway seems like the obvious thing to
do, so somebody must have done it already?
Also, are there any anonymous nntp servers which function as archives
for some or all of the groups, ie have long-term storage available by
nntp? CD-ROM online or something. With the www news gateway and some
minor modifications for some purposes saving an interesting article to
local disk would be unnecssary, instead just the message ID would be
saved (or added to a node on an organized www document so one could
find it later) and the article could later be found on a generic nntp
archive. To save bandwidth and avoid single points of failure the
nntp archive could be a distributed database with servers on various
parts of the world and cooperation and caching between them - the
servers could compute a hash from the message ID and archive a certain
percentage of the article. With some coordination the burden of disk
space could be shared so there needn't be one gigantic nntp archive.
//Jyrki