W3C home > Mailing lists > Public > www-talk@w3.org > May to June 1995

Re: Byte ranges (actually robots)

From: Gavin Nicol <gtn@ebt.com>
Date: Thu, 25 May 1995 09:21:10 -0400
Message-Id: <199505251321.JAA13703@ebt-inc.ebt.com>
To: nazgul@utopia.com
Cc: www-talk@www10.w3.org

>So what to do? If we recognize the user-agent as a known robot, we give it
>a flat presentation of the structure.  The catch of course, is that there
>is no standard way to recognize a robot.  Some convention in the UserAgent
>field would be sufficient, but....

Isn't there a standard permissions file that robots can fetch?
Received on Thursday, 25 May 1995 09:18:54 UTC

This archive was generated by hypermail 2.4.0 : Monday, 20 January 2020 16:08:17 UTC