Re: Thougt paper: Automatic web characterisation

From: Chris Roadknight (
Date: Mon, Dec 07 1998

From: Chris Roadknight <>
To: www-wca <>
Date: Mon, 7 Dec 1998 11:45:26 -0000
Message-ID: <01be21d7$14e0e1c0$>
Subject: Re: Thougt paper: Automatic web characterisation

A quick comment re:section 4 of Volker's remarks

Do we really want to remove robots from any analysis?
I agree that the identification of robots is important but removing 
all non-human requests from cache log based analysis seems
a very error prone task as It involves identifying robots, automatic
subscriptions, robot assisted browsing and linked/embedded objects.

I would be grateful for any opinions on favoured methods for filtering
human requests from automated requests.

Chris Roadknight
BT Research

>I have gathered a few thoughts around the topic discussed in J. Hjelms
>paper on an automatic web characterisation architecture and put them on
>the web:
>volker turau
>FH Wiesbaden Fachbereich Informatik 
>Tel.: +49-611-9495-205 FAX +49-611-9495-210