Newsgroup list cache - error in design

------- Forwarded Message

Return-Path: puzio@zodiac1.mimuw.edu.pl 
Return-Path: puzio@zodiac1.mimuw.edu.pl
Received: from www10.w3.org (www10.w3.org [18.23.0.20]) by anansi.w3.org 
(8.7.1/8.7.1) with SMTP id KAA02364 for <frystyk@anansi.w3.org>; Mon, 15 Apr 
1996 10:42:11 -0400 (EDT)
Received: from zodiac1.mimuw.edu.pl by www10.w3.org (5.0/NSCS-1.0S) 
	id AA10513; Mon, 15 Apr 1996 10:41:50 +0500
Received: from pc4180a.mimuw.edu.pl (pc4180a.mimuw.edu.pl [148.81.12.160]) by 
zodiac1.mimuw.edu.pl (8.6.12/8.6.9) with SMTP id QAA03489 for <frystyk@w3.org>
; Mon, 15 Apr 1996 16:42:40 +0200
Received: by pc4180a.mimuw.edu.pl with Microsoft Mail
	id <01BB2AEC.29D69DC0@pc4180a.mimuw.edu.pl>; Mon, 15 Apr 1996 16:54:08 +-200
Message-Id: <01BB2AEC.29D69DC0@pc4180a.mimuw.edu.pl>
From: Maciej Puzio <puzio@zodiac1.mimuw.edu.pl>
To: "'Henrik Frystyk Nielsen'" <frystyk@w3.org>
Subject: Newsgroup list cache - error in design
Date: Mon, 15 Apr 1996 16:54:05 +-200
Return-Receipt-To: <puzio@zodiac1.mimuw.edu.pl>
Mime-Version: 1.0
Content-Type: multipart/mixed; boundary="---- =_NextPart_000_01BB2AEC.29F08E60"


- ------ =_NextPart_000_01BB2AEC.29F08E60
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit

Hi Henrik

> Have a nice weekend

Thanks a lot, I had a nice weekend during which I have discovered an error in 
design
of the newsgroup list cache (piece of code in HTNewsLs.c written by me...). 
Mea culpa!

The problem with the problem is that this piece of code works very well in 99% 
of 
situations. (In fact, it worked in 100% of situations when I tested it.) But 
the fault
in the design is evident. I'm working on this now and I hope I will soon send 
you
corrected version.

Short description of the newsgroups list cache

The cache is intended for storing list of the newsgroups availables on a given 
NEWS
server.
Currently the cache is very simple and can handle only one NEWS host at a 
time. 
It has 5 main operations:
- - HTNewsCache_init - initializes the cache to hold the data from the given 
host
- - HTNewsCache_add - adds a line from the newsgroup list to the cache
- - HTNewsCache_cachesHost - checks whether cache holds data from the given 
host
- - HTNewsCache_parse - parses all cached data
- - HTNewsCache_delete - deletes the cache
(HTNewsCache_requestToHost is a helper function which gets the NEWS host name
from the request)

In HTNewsLs.c _init and _add are called to put the newsgroup names into the 
cache.
In HTNews.c _cachesHost and _parse are called to read the newsgroup names from
the cache instead of the network. This is simple, but in some situations not 
correct.
If there are two simultaneous requests for the newsgroups list, and the 
requested data
is not cached, both request threads can start adding data to the cache (if the 
host is
different) or one of them can parse not yet complete cache data (if the host 
is the same
in both requests). (Simply I have forgotten about multithreading issues...)

I was thinking during the weekend on the problem and came to the conclusion 
that
writing to a cache should be accomplished in the same way as writing to files 
with
exclusive access. NEWS host names would play the role of filenames.
That is, if we want to write to a cache, we call HTNewsCache_open(news host 
name)
and get a handle (special cache handle). Then we write to the cache using this 
handle
and after everything is done, we close the handle. If another thread is 
writing the data
of the same host at the same time, open fails. HTNewsCache_cachesHost returns
the handle of the data or NULL, and we pass this handle to the 
HTNewsCache_parse.
The latter action is atomic (can't be broken by another thread), but perhaps 
in the
future this should be handled differently. So perhaps it would be better to use
open/close for cache reading also? The simultaneous reads would be allowed, but
not writes interleaving reads. This design would also allow for managing 
multiple 
NEWS hosts.

Nice idea, isn't it? If we follow this way we will implement the whole 
internal filesystem
soon. And everything is only for speed up one kind of NEWS lists a little bit! 
Perhaps
it'll be better to use some kind of memory files (there is such a thing in 
Windows 95/NT,
I suppose in UNIX also)? Or implement memory files as one of helper modules in 
the
library, but in the generic way (not only for NEWS cache)?

What do you think about this?

Best Regards

Maciej Puzio
puzio@laser.mimuw.edu.pl












- ------ =_NextPart_000_01BB2AEC.29F08E60
Content-Type: application/ms-tnef
Content-Transfer-Encoding: base64

eJ8+IggOAQaQCAAEAAAAAAABAAEAAQeQBgAIAAAA4gQAAAAAAADmAAENgAQAAgAAAAIAAgABBJAG
ABABAAABAAAADAAAAAMAADADAAAACwAPDgAAAAACAf8PAQAAAEMAAAAAAAAAgSsfpL6jEBmdbgDd
AQ9UAgAAAABIZW5yaWsgRnJ5c3R5ayBOaWVsc2VuAFNNVFAAZnJ5c3R5a0B3My5vcmcAAB4AAjAB
AAAABQAAAFNNVFAAAAAAHgADMAEAAAAPAAAAZnJ5c3R5a0B3My5vcmcAAAMAFQwBAAAAAwD+DwYA
AAAeAAEwAQAAABkAAAAnSGVucmlrIEZyeXN0eWsgTmllbHNlbicAAAAAAgELMAEAAAAUAAAAU01U
UDpGUllTVFlLQFczLk9SRwADAAA5AAAAAAsAQDoBAAAAAgH2DwEAAAAEAAAAAAAAA8UvAQiABwAY
AAAASVBNLk1pY3Jvc29mdCBNYWlsLk5vdGUAMQgBBIABACcAAABOZXdzZ3JvdXAgbGlzdCBjYWNo
ZSAtIGVycm9yIGluIGRlc2lnbgDiDQEFgAMADgAAAMwHBAAPABAANgAFAAEAMgEBIIADAA4AAADM
BwQADwAQAAoALgABAC8BAQmAAQAhAAAAREZDRDdEM0JEMzk2Q0YxMUIyODQwMjYwOENBNUI3M0IA
WQcBA5AGAMAJAAASAAAACwAjAAEAAAADACYAAAAAAAsAKQABAAAAAwA2AAAAAABAADkAgNk9ZNsq
uwEeAHAAAQAAACcAAABOZXdzZ3JvdXAgbGlzdCBjYWNoZSAtIGVycm9yIGluIGRlc2lnbgAAAgFx
AAEAAAAWAAAAAbsq22QtO33N4JbTEc+yhAJgjKW3OwAAHgAeDAEAAAAFAAAAU01UUAAAAAAeAB8M
AQAAABkAAABwdXppb0BsYXNlci5taW11dy5lZHUucGwAAAAAAwAGEJA+fCMDAAcQEQoAAB4ACBAB
AAAAZQAAAEhJSEVOUklLSEFWRUFOSUNFV0VFS0VORFRIQU5LU0FMT1QsSUhBREFOSUNFV0VFS0VO
RERVUklOR1dISUNISUhBVkVESVNDT1ZFUkVEQU5FUlJPUklOREVTSUdOT0ZUSEVORVcAAAAAAgEJ
EAEAAAA0CAAAMAgAADcQAABMWkZ1m6114/8ACgEPAhUCqAXrAoMAUALyCQIAY2gKwHNldDL1AAAq
AuFhB4AGAAbDAoBsTVQShwHxMgPFAgBwTHJxE/ERdjM4EoFz6HRlbQKDMxENFXAR+KEP3UhlbHYC
gzQWb48R+AcTAoAHEyBDRQKDzjUUDxo/G099fQqACM/FCdk7IL8yNTUCgAqBgw2xC2BuZzEwNByQ
FwsKHJIMAWMAQCBIabclkAnwBRBrCoUKhT4K4a8LZBmRC/IUkG8V0GMFQEhIYXYScGEgAwBj1RJw
dwngawnwZAqHJJ/lCoVUEYBuawQgKPAgUKB0LCBJIBGAZCjtXCBkCHEjwClQaCkgaPsssyjBZAQA
BaAowCDBKOB/A6AEkANgBcALgC4AB5BpBGduCoVvZiB0aLUScG4H0HMJwAhgcCxgmwQABUBjANAx
0ShwCJCfKTExkQWgDbAwgkhUB8Hgc0xzLmMpUAUQAkChCfAgYnkgB4AuNfAkKS4mlk1lKPBjdfps
CrAhJjwr4BJwKCECYP8V4ClQNUAuwDHCOIYEADGx3mEFQDHAOhEzbHcFsCwhNy+hNbApYGwDIDCR
OTlOJTGCCoUAkHR1OmBp1wIgNOAzQEkDoGYA0CyR3zVAO7Mv0TCRI+AwPRM92L8ucTVxLMAV0BXB
P7F0NhB4IEJ1OnIScD7QNxB0HwqFMJExwjDEOgJldmnnDbACMDYwSSc44TvRLkLzAiA6hG5vB+AA
cC0ALME8b3AScCzAA/A8kXNv90YBEbAt4XkIYAqFBaEgwF8ocC/RL6EAkAIgLiY8U/9HIAAgMLIF
Az4xMY8EIDKo/zdvMuU6EQuANWENsC0AAhB9PbF0BbAuQjKjTE8o4HY/C3ALYDixBCBGASjwZ2nD
KMADoE5FV1M9ZgSQuy+hSjZDCHAgwAIwbDWw7zHCTyc8IwCQbQtQKNEt4fcy8AOgK/FkV2ECIFXR
AiD/EnBT0kcRMsE6YSjwPiA10d0mlkkFQBGABCA1NcALcatRQEdAcj4UOgqFLTRl+kMzAl8LgD8x
XNBdsgcxvGl6B5FWCFCQRxFsLQD/Q9M6YCjwA1Ixs1N0WWJcb/9dcizwLQBc0GMRLDMLgELBf2Cm
Mg1fgVYHYc9dcjLzc+5IWXJc0DMRYywhQTFkgf9JATMDX7IEIGBPYV9dCQqx/xGwXMFtEywxPJEy
8y3xalH/a89dcg2wOMAV0FzBcEReyW0KhShc+iDAcQpQFcBUfm9oIzoRKPBkkDcgaVFm3HVuKHBM
Ai6EZxHAXsTfWRgSQgqFapdzJSkmPD6h/zR5XaRGwmMDCsBPAjyQL9H9X4FwQnUyCBJCT5NlyUo2
/3mHejJnyXrDbRR7XCDALPH/fH8HkQNSCoVWCQCAFdAs8f0xl3Q7wjYwK+BPggQgVyT9LKBiQnEw
kUfgEmFAiUaA/zLRSTR+lzGje2F7UoYRVxL7QwEAcGUIYAQgcyWDkQWx/0x/FcAsoEbCd+lujEZT
MtX+ZIdhKEAuwHMlZHGCIgQg/1fSFcAKwFmRYyAuQmpDZbr/M0AGkGRzWWMEAAqFL1AN0P8vsQIw
QkAFsVjSMZQ44FfS+20UiPJ5EcAz4VdCcIEy9HdqQ5RtZHNzdvgwkZDqc3k2IShTVzI1sC7lUEFn
/yhANWIBoAhgBUCLUjkRgiL/LkIEAXNRNfJ4rSlQWxE6kf8sEJMDLiRkgi2WRgM4aFeU9xJhZbYC
IGMKQEnyOjMKhd81IqJjX5A24TMDc0cgNxD1LQBiKNFjmIMEAG5yQ6XfmwKhUTWwWxGmuWYDEAeR
8zkCCoVleKWDKMKokAeQ/z5hdkxBEagDC1FV5ANgWFL/MaCrMn1TSjYr4XPiPxExoP8pYKFRAjBf
cjUiX2OnZSygz7Ghe5Jc60cxbigyAnaIn3iWRsJ1sXQiWCQoc0dA/mMb4mmFWCM2IThBdTEpQX+y
ZlYIpaGiczoRWBS1+WH/AYBpUUSwPDGhop+iLgBY0f+zVCBQbUGUo1gjRSExoABw/5DxaVGRtDoC
prdgFTEsqcP/WWaph1oCLKC0sj7BAxA+Yf9nD2gUIMA98ASglUa+mFFGw2pDBbFOVUxMjgSxof8K
sAQRuwllpmxfsGgScAtg3zVRtkF09HQCUJBtKSAzQO1X0ScFQKhRYgNgKZE1kv2/jCmHZFvBEYBN
MUOkdyb/QnAIcL5yhtKn91gULfGV1vtV0DYwU3wR0TY/QqgVqFDfzTNfgYvQTfa0si++NFBC7zL0
n0YHQEfgPzgziz2R8v/WB24BRpCQo0JwCoWI8rJT/U+UcjjAKLAuQpHThmVEFd+uZNmi2/RQMwOB
YVNwLlH/ntNXUgqFWRewV+JWKSJE0fphPxFzzsI1QNngiiGxof8CEOCjOpOqErGiR6JXMweA/7Hy
orJfsTQivIESQAMgqzPvFbM9ZkfxNjBBLeG8zFiD/1BDR0Av0TJxWNJFsYVjWRMvMqJjpAJAV2Fi
NUAhIOZQ0TRDNnQnPJHWbofk3+1GB4AEYDxBqzQoilSG0t51LrFZ4b0kA6BX7VFGkIEEIDk1L05U
LKDXGZ/gcHC+UjCRVU5J6ljZkynZ4E8wceeX8nv/WxGWpXRVBGE3EH2D0ckyoPvPMArAeYdnauMJ
8AZxNQH/qiG08IkB6+dZEzLz95AmPP5XOlK9oEhyoZSeZTqS/13iQpFyUmVnCxGVRjZWy6zgCJBq
70B1ej4wCoXbfDAFIUALYFSBLs5Qi1Dkdy4JgHUuC1AmPAe//wjPCd8p+ye9Cs4kTwyOIzECABBg
AwAQEAAAAAADABEQAAAAAEAABzCgPRNX1Sq7AUAACDCgPRNX1Sq7AR4APQABAAAAAQAAAAAAAADi
eg==

- ------ =_NextPart_000_01BB2AEC.29F08E60--

------- End of Forwarded Message


-- 

Henrik Frystyk Nielsen, <frystyk@w3.org>
World-Wide Web Consortium, MIT/LCS NE43-356
545 Technology Square, Cambridge MA 02139, USA

Received on Friday, 3 May 1996 14:35:54 UTC