Re: Crawlers and WebID

Richard  Cyganiak kindly moved the part I wrote up to its own space

   http://www.w3.org/wiki/WebID_and_Crawler

after reformatting the text.

	Henry

On 23 Jun 2011, at 18:50, Henry Story wrote:

> There was a very long discussion on the Linked Data mailing list on issues of student build crawlers going crazy on linked data sites, in such a way that it amounted to a denial of service attacks.
> 
> The thread is entitled "Think before you write Semantic Web crawlers"
> 
>   http://lists.w3.org/Archives/Public/public-lod/2011Jun/0499.html
> 
> Kingsley and I have been very vocal in urging for WebID enabled crawlers and linked data publishers to solve this problem. I think we even got some good mind share on it.
> 
> I wrote up the details here:
> 
>  http://www.w3.org/wiki/Bad_Crawlers
> 
> We could use this as a powerful use case for WebID. Please add improvements on that page, and we should link to it somewhere.
> 
>  Henry
> 
> 
> 
> Social Web Architect
> http://bblfish.net/
> 

Social Web Architect
http://bblfish.net/

Received on Thursday, 23 June 2011 21:59:05 UTC