Re: Robots
From: Jim Pitkow (pitkow@parc.xerox.com)
Date: Fri, Dec 11 1998
Date: Fri, 11 Dec 1998 10:08:21 PST
To: www-wca <www-wca@w3.org>
From: Jim Pitkow <pitkow@parc.xerox.com>
Message-Id: <98Dec11.100937pst."147472"@mailback.parc.xerox.com>
Subject: Re: Robots
At 09:23 AM 12/10/98 , Henrik Frystyk Nielsen wrote:
>
> I am aware of one such list which is linked off
>
> <http://www.w3.org/Protocols/>http://www.w3.org/Protocols/
I'm aware of that page, which while providing a nice list of robots, also
shares some similar conclusions:
Note that now robot technology is being used in increasing numbers of end-user
products, this list is becoming less useful and complete.
which leaves us with the same mess.
>
> be better to characterize browsing patterns than user agents. There are many
> ways browsing can be done:
>
> GUI browsing
> line mode browsing
> high bandwidth vs. low bandwidth browsing
> voice dictated browsing
> link checking and content validation
> mirror generating
> indexing (breath or depth)
> cache validation
> ...
>
> all of which I think are valid ways of using the Web but are likely to
result
> in different browsing patterns. I also think that it is likely that the same
> application can cover several modes and hence splitting it by user-agent may
> not capture the various modes of browsing.
The examples above are interesting in that they capture intentionality, which
if I think we could capture, would be great, but it's a really difficult
task... I was more concerned initially with just the 'whose following the
links' aspect, so that different paths could be processed separately..