- From: <bugzilla@wiggum.w3.org>
- Date: Wed, 06 Oct 2004 02:00:00 +0000
- To: www-validator-cvs@w3.org
http://www.w3.org/Bugs/Public/show_bug.cgi?id=893
ot@w3.org changed:
What |Removed |Added
----------------------------------------------------------------------------
AssignedTo|ot@w3.org |ville.skytta@iki.fi
Component|Other |checklink
Product|CSSValidator |LinkChecker
Summary|excessive robot.txt probes |cache (non) existence
Version|CSS Validator |4.0
------- Additional Comments From ot@w3.org 2004-10-06 02:00 -------
Most likely this is the Link Checker doing this, not the CSS validator.
Our site is not "probing" yours.
Someone (possibly someone local to you, or someone with a site linking to yours) is certainly checking
links to your site, and the link checker is following the robots exclusion protocol and doing your server
a favor in doing so.
That said, a possible enhancement would be that the link checker cache the existence/lack of robots.txt
for a given site instead of querying for it again and again.
Reassigning to proper product and owner.
------- You are receiving this mail because: -------
You are the QA contact for the bug, or are watching the QA contact.
Received on Wednesday, 6 October 2004 02:00:01 UTC