- From: Paul <paul@ianlynam.com>
- Date: Mon, 2 Jan 2012 12:28:09 +0900
- To: www-validator@w3.org
The link validator has a built-in one second delay between page fetches - I appreciate the reasons but when I am trying to check my own site it can take a while. As I don't care how fast you crawl my server at my request can I suggest the following: Check the root level for a file (maybe use robots.txt?) for a 'crawl-delay' value and if it's zero let the validator run at full speed. plus: Site admins can then use the same process to block the validator if they didn't request it. plus: minor stress test
Received on Monday, 2 January 2012 14:45:43 UTC