[Prev][Next][Index][Thread]

Re: wcol-beta05, announce and call for teseter



You (Kenichi Chinen) write:

>Wcol is prefetching HTTP proxy server. 
>Prefetch, get data before user's request is possible to reduce
>response time. Wcol apply this technique to linked data in WWW.

It looks promising, and few people have tried this before.
(I sometimes prefetch by hand, using a robot :-)

>So, Your clients will make fast clients via wcol.

I'm not so sure.  Suppose it tried to maintain a lookahead of one: it would
prefetch every document linked to from a document already fetched by some
client.  Let's say that 20% of the links on a page is actually followed by
the user.  This roughly leads to 20% overhead: in disk space, in processing
time on the proxy, and in network load between the proxy and the world.
Especially disk space and network load are common bottlenecks; they tend to
be used to full capacity.  Hence, savings will have to be made elsewhere; in
the case of disk space, by expiring documents more quickly.  As a result, the
cache will be less effective (allocating the same amount of disk space, its
hit rate will drop to approximately 20% of what it was).  The same effect
occurs for network load, if the network is saturated.

The same is true for Netscape's multiple connections, of course.

The story is a little different if you do prefetching off-line, at off-peak
hours; in that case, the disk space problem will be worse, and the network
load problem will be less of a problem.

Also, you can try to be smart about what to prefetch and when, and you can
analyse the effects of prefetching.  I hope that this will be done, that it
will lead to some interesting results on WWW usage patterns and on the
effects of prefetching.  Wcol will be an excellent tool to do it!

>Documents (http://shika.aist-nara.ac.jp/products/wcol/wcol.html) is
>under construction.

Are you planning to add some documentation on comparisons with other proxy
products, and on the exact prefetching algorithm you use?  Why have you
chosen to develop your own proxy server?

(BTW, the conversion feature is very nice.  Do you support content
negotiation, as supported by the Apache httpd?)

-- 
Reinier Post						 reinpost@win.tue.nl
a.k.a. <A HREF="http://www.win.tue.nl/win/cs/is/reinpost/">me</A>
[LINK] [LINK] [LINK] [LINK] [LINK] [LINK] [LINK] [LINK] [LINK] [LINK] [LINK]

Follow-Ups: References: