Re: Log files and sizing the server

From: Jim Pitkow (pitkow@parc.xerox.com)
Date: Wed, Nov 03 1999


Date: Tue, 2 Nov 1999 23:38:39 PST
To: "'www-wca@w3.org'" <www-wca@w3.org>
From: Jim Pitkow <pitkow@parc.xerox.com>
Message-Id: <99Nov2.234514pst."360123"@louise.parc.xerox.com>
Subject: Re: Log files and sizing the server


35 minutes for 60 Mb file is slow - 1.6Mb per minute!  Brad's number of
40Mb in 15 seconds is more realistic - 2.6 Mb per second (and good disks
can read an order of magnitude faster).  Maybe check your system to make
sure it is ok.?

For what it's worth, we analyze 10's of Gigabytes of log file data on so-so
hardware at a time easily within a few hours, though a lot depends upon the
complexity of information we extract.  We've used webalizer - it's true to
it's claims - fast.

At 03:30 PM 11/2/99 , Neff, Robert wrote:
>Has anyone done any analysis with log files and report generation?  We are
>looking to try to size a server and find a quad-pentium takes approx 35
>minute for a 60 Mb file (one day's log).  Where a month, say averaging 18000
>MB of data - we get errors.  We are using webtrends and are looking for
>other packages and options.
>
>Any ideas?
>
>Robert Neff
>