- From: Bjoern Hoehrmann <derhoermi@gmx.net>
- Date: Wed, 09 Jun 2004 00:06:49 +0200
- To: Ville Skyttä <ville.skytta@iki.fi>
- Cc: QA Dev <public-qa-dev@w3.org>
* Ville Skyttä wrote: >Regarding the beta announcement, I would like to request feedback in >particular of the following (feel free to rephrase, comment, and add >items): > >- Robots exclusion standard support in the link checker. Good or bad? It would be good to announce in what way it supports it; as discussed, there are apparently several approaches, like do you read the submitted document if the robots.txt of that server does not allow it or will it refuse to check any link in that case (as it does not download anything but robots.txt). Without such information one would have to figure this out by testing or looking at the code which is unlikely to yield in much feedback.
Received on Tuesday, 8 June 2004 18:07:24 UTC