Re: New response code

At 08:36 am 2/19/96 -0800, Shel Kaphan spake:
>Some applications would generate pages differently if they are being
>probed by a robot.  For instance, in applications that use URL

>So, I'd like to propose that robots be allowed to identify themselves
>as such by including a simple header line in requests, which ought to
>be passed along to CGI programs.  The header could just be "robot: true"

   I don't see that this is necessarily an issue for HTTP. If you 
are interested in pursuing it, I would suggest that you take it up 
with the Robots mailing list. Here's their subscription info: 

        "To subscribe to this list, send a mail message to 
         robots-request@webcrawler.com, with the word subscribe 
         on the first line of the body."

   If all those folks find they have a pressing need for a new 
header field in HTTP, you may then have the necessary leverage over 
here. OTOH, I think your problem could easily be solved if the 
robot authors would agree to put the word "ROBOT" somewhere in their 
User-Agent headers. It seems just as effective as creating a new 
header field, and you would need their cooperation either way. 


+--------------------------------------------------------------------------+
| BearHeart / Bill Weinman | BearHeart@bearnet.com | http://www.bearnet.com/ 
| Author of The CGI Book -- http://www.bearnet.com/cgibook/