W3C home > Mailing lists > Public > www-tag@w3.org > May 2011

Re: Changing the meaning of Robots.txt

From: Karl Dubost <karld@opera.com>
Date: Tue, 10 May 2011 11:10:36 -0400
Message-Id: <0E524B4D-749B-41B6-8E62-D0A638554A71@opera.com>
To: "www-tag@w3.org List" <www-tag@w3.org>

Le 10 mai 2011 à 10:13, Karl Dubost a écrit :
> On Archive.org wiki, there was a Web page about robots.txt [1] created by JSCott [2]

ooops Archiveteam wiki which is different from archive.org.
Thanks for people who told me off the record.

> The title is  "ROBOTS.TXT IS A SUICIDE NOTE"
> 
>    ROBOTS.TXT is a stupid, silly idea in the modern era.
>    Archive Team entirely ignores it and with precisely one
>    exception, everyone else should too.
> 
>    If you do not know what ROBOTS.TXT is and you run a
>    site... excellent. If you do know what it is and you
>    have one, delete it. Regardless, Archive Team will
>    ignore it and we'll delete your complaints, just like
>    you should be deleting ROBOTS.TXT.
> 
> My big issue with the rant is not necessary the will to kill robots.txt but more exactly to change the expectation that people had with regards to this protocol. I expressed some of my concerns in the Discuss page [3] of the wiki.
> 
> 
> [1]: http://www.archiveteam.org/index.php?title=Robots.txt
> [2]: http://www.archiveteam.org/index.php?title=User:Jscott
> [3]: http://www.archiveteam.org/index.php?title=Talk:Robots.txt
> 
> -- 
> Karl Dubost - http://dev.opera.com/
> Developer Relations & Tools, Opera Software
> 
> 

-- 
Karl Dubost - http://dev.opera.com/
Developer Relations & Tools, Opera Software
Received on Tuesday, 10 May 2011 15:11:06 GMT

This archive was generated by hypermail 2.2.0+W3C-0.50 : Thursday, 26 April 2012 12:48:35 GMT