W3C home > Mailing lists > Public > site-comments@w3.org > July 2012

robots.txt on dvcs.w3.org and w3.org

From: Simon Pieters <simonp@opera.com>
Date: Wed, 11 Jul 2012 13:50:18 +0200
To: "site-comments@w3.org" <site-comments@w3.org>
Message-ID: <op.wg95t4svidj3kv@simons-macbook-pro.local>

http://dvcs.w3.org/robots.txt disallows all robots except W3C-checklink.  
http://www.w3.org/robots.txt disallows all robots for /Bugs/. This makes  
it hard to find specs on dvcs.w3.org and bugs on w3.org/Bugs/ with search  
engines. Can this be changed, please?

Simon Pieters
Opera Software
Received on Wednesday, 11 July 2012 11:50:54 UTC

This archive was generated by hypermail 2.3.1 : Tuesday, 6 January 2015 21:15:44 UTC