I'll say. That spider option shouldn't check any additional pages since http://www.planetsvg.com/robots.txt says User-agent: * Disallow: / Rob Russell rob@latenightpc.com http://www.latenightpc.com/blog Ruud Steltenpool wrote: > > http://htmlhelp.com/cgi-bin/validate.cgi?url=http%3A%2F%2Fwww.planetsvg.com%2F&warnings=yes&spider=yes&hidevalid=yes > >Received on Monday, 24 November 2008 15:27:22 GMT
This archive was generated by hypermail 2.2.0+W3C-0.50 : Tuesday, 14 April 2009 16:29:30 GMT