Re: Robots

From: Jim Pitkow (pitkow@parc.xerox.com)
Date: Fri, Dec 11 1998


Date: Fri, 11 Dec 1998 10:08:21 PST
To: www-wca <www-wca@w3.org>
From: Jim Pitkow <pitkow@parc.xerox.com>
Message-Id: <98Dec11.100937pst."147472"@mailback.parc.xerox.com>
Subject: Re: Robots

At 09:23 AM 12/10/98 , Henrik Frystyk Nielsen wrote: 

>
> I am aware of one such list which is linked off 
>
>         <http://www.w3.org/Protocols/>http://www.w3.org/Protocols/ 


I'm aware of that page, which while providing a nice list of robots, also
shares some similar conclusions: 

Note that now robot technology is being used in increasing numbers of end-user
products, this list is becoming less useful and complete. 

which leaves us with the same mess.

>
> be better to characterize browsing patterns than user agents. There are many
> ways browsing can be done: 
>
>         GUI browsing 
>         line mode browsing 
>         high bandwidth vs. low bandwidth browsing 
>         voice dictated browsing 
>         link checking and content validation 
>         mirror generating 
>         indexing (breath or depth) 
>         cache validation 
>         ... 
>
> all of which I think are valid ways of using the Web but are likely to
result
> in different browsing patterns. I also think that it is likely that the same
> application can cover several modes and hence splitting it by user-agent may
> not capture the various modes of browsing. 


The examples above are interesting in that they capture intentionality, which
if I think we could capture, would be great, but it's a really difficult
task...  I was more concerned initially with just the 'whose following the
links' aspect, so that different paths could be processed separately..