Re: Changing the meaning of Robots.txt

Le 10 mai 2011 à 10:13, Karl Dubost a écrit :
> On Archive.org wiki, there was a Web page about robots.txt [1] created by JSCott [2]

ooops Archiveteam wiki which is different from archive.org.
Thanks for people who told me off the record.

> The title is  "ROBOTS.TXT IS A SUICIDE NOTE"
> 
>    ROBOTS.TXT is a stupid, silly idea in the modern era.
>    Archive Team entirely ignores it and with precisely one
>    exception, everyone else should too.
> 
>    If you do not know what ROBOTS.TXT is and you run a
>    site... excellent. If you do know what it is and you
>    have one, delete it. Regardless, Archive Team will
>    ignore it and we'll delete your complaints, just like
>    you should be deleting ROBOTS.TXT.
> 
> My big issue with the rant is not necessary the will to kill robots.txt but more exactly to change the expectation that people had with regards to this protocol. I expressed some of my concerns in the Discuss page [3] of the wiki.
> 
> 
> [1]: http://www.archiveteam.org/index.php?title=Robots.txt
> [2]: http://www.archiveteam.org/index.php?title=User:Jscott
> [3]: http://www.archiveteam.org/index.php?title=Talk:Robots.txt
> 
> -- 
> Karl Dubost - http://dev.opera.com/
> Developer Relations & Tools, Opera Software
> 
> 

-- 
Karl Dubost - http://dev.opera.com/
Developer Relations & Tools, Opera Software

Received on Tuesday, 10 May 2011 15:11:06 UTC