W3C home > Mailing lists > Public > site-comments@w3.org > July 2012

robots.txt on dvcs.w3.org and w3.org

From: Simon Pieters <simonp@opera.com>
Date: Wed, 11 Jul 2012 13:50:18 +0200
To: "site-comments@w3.org" <site-comments@w3.org>
Message-ID: <op.wg95t4svidj3kv@simons-macbook-pro.local>
Hi,

http://dvcs.w3.org/robots.txt disallows all robots except W3C-checklink.  
http://www.w3.org/robots.txt disallows all robots for /Bugs/. This makes  
it hard to find specs on dvcs.w3.org and bugs on w3.org/Bugs/ with search  
engines. Can this be changed, please?

-- 
Simon Pieters
Opera Software
Received on Wednesday, 11 July 2012 11:50:54 GMT

This archive was generated by hypermail 2.2.0+W3C-0.50 : Wednesday, 24 October 2012 16:21:34 GMT