Robots

From: Volker Turau (turau@informatik.fh-wiesbaden.de)
Date: Mon, Dec 07 1998


Date: Mon, 7 Dec 1998 15:04:37 +0100 (MET)
From: Volker Turau <turau@informatik.fh-wiesbaden.de>
To: Chris Roadknight <roadknic@drake.bt.co.uk>
cc: www-wca <www-wca@w3.org>
Message-ID: <Pine.GSO.3.96.981207133042.26434A-100000@pcsrv1.informatik.fh-wiesbaden.de>
Subject: Robots



Hi Chris,

this is a misunderstanding, what I mean is that robots show a behavior
that is completely different from human behavior and therefore it must be
treated separately if we want to analyze user behavior. Filtering out
human requests is a difficult if not impossible task. In my
analysis I use a crude heuristic: every user agent different from Mozilla
is considered a robot. I know that there have been cases that robots used
Mozilla as a name in the USER_AGENT header and I know that there are many
more browsers besides Netscape and MSIE. But it gives me a good estimate.

volker 

volker turau
FH Wiesbaden Fachbereich Informatik 
Tel.: +49-611-9495-205 FAX +49-611-9495-210
http://www.informatik.fh-wiesbaden.de/~turau