Dom, On Apr 21, 2004, at 10:55, olivier Thereaux wrote: > > * 1 - checklink * > There is, however, dissent on whether/when the link checker is > actually a robot, and a feeling (for some of us) that following > robots.txt makes it less useful. A first workaround (implemented > during the meeting!) is to add a note explaining how to modify one's > robots.txt to allow the link checker to do more than other agents. The note is at: http://qa-dev.w3.org/wlc/#bot Could I assign you the (extra) action item of taking care of this for the robots.txt at www.w3.org? This would be setting a good example. Thanks -- olivierReceived on Tuesday, 20 April 2004 22:02:19 GMT
This archive was generated by hypermail 2.2.0+W3C-0.50 : Thursday, 19 August 2010 18:12:44 GMT