- From: Paul Prescod <paul@prescod.net>
- Date: Tue, 18 Feb 2003 17:25:51 -0800
- To: www-tag@w3.org
Patrick.Stickler@nokia.com wrote: > > .... > A specific question to help me determine that: If the server owner > says "no crawlers at all on this server" and a tenant says "all my > own content can be crawled", should the tenant's content be crawled? The server owner runs the HTTP server. No data can go in or out of the system without their explicit or implicit agreement. In particular, they can trivially block the MGET method or metadata header or whatever emerges in their Apache configuration. Paul Prescod
Received on Tuesday, 18 February 2003 20:26:05 UTC