robots.txt on dvcs.w3.org and w3.org from Simon Pieters on 2012-07-11 (site-comments@w3.org from July 2012)

From: Simon Pieters <simonp@opera.com>
Date: Wed, 11 Jul 2012 13:50:18 +0200
To: "site-comments@w3.org" <site-comments@w3.org>
Message-ID: <op.wg95t4svidj3kv@simons-macbook-pro.local>

Hi,

http://dvcs.w3.org/robots.txt disallows all robots except W3C-checklink.  
http://www.w3.org/robots.txt disallows all robots for /Bugs/. This makes  
it hard to find specs on dvcs.w3.org and bugs on w3.org/Bugs/ with search  
engines. Can this be changed, please?

-- 
Simon Pieters
Opera Software

Received on Wednesday, 11 July 2012 11:50:54 UTC