Alex Hopmann: Re: New response code

[ meant for the list -- ange ]

------- Forwarded Message

Date:    Mon, 19 Feb 1996 10:17:11 -0800
From:    hopmann@holonet.net (Alex Hopmann)
To:      http-wg-request@cuckoo.hpl.hp.com
Subject: Re: New response code

Shel Kaphan wrote:
>Some applications would generate pages differently if they are being
>probed by a robot.  For instance, in applications that use URL
>encoding of session information (which will be with us until cookies
>take over completely)  it might be preferable not to generate session
>ids, or at least not new ones, for robots.
My hunch is this is a bad idea. In general sites should exclude those URLs
that encode session information from Robots. And the concept of servers
returning different results to robots is open to abuse- There already are
some sites that were designed to detect the robots from the more popular
sites and returned some sort of document designed to match everything. This
way sites can lure people to their sites without having anything of interest
to the browser. Because of this in practice the robot authors are probably
not going to want to identify themselves anyway (beyond fetching the
"robots.txt" file).
Alex Hopmann
ResNova Software, Inc.
hopmann@holonet.net


------- End of Forwarded Message

Received on Tuesday, 20 February 1996 05:55:07 UTC