W3C home > Mailing lists > Public > www-talk@w3.org > May to June 1995

Re: Byte ranges (actually robots)

From: Darren New <dnew@sgf.fv.com>
Date: Thu, 25 May 1995 11:44:08 +0100
To: Kee Hinckley <nazgul@utopia.com>
Cc: Multiple recipients of list <www-talk@www10.w3.org>
Message-Id: <Pine.3.89.9505251146.B1315-0100000@sgf.fv.com>
> and I can't just flag anyone who fetches robot.txt as a robot, since of
> course a person may fetch it too.

And of course not all robots will fetch robot.txt, either.  Welcome to 
the internet!  :-)
Received on Thursday, 25 May 1995 11:48:31 UTC

This archive was generated by hypermail 2.4.0 : Monday, 20 January 2020 16:08:17 UTC