W3C home > Mailing lists > Public > public-lod@w3.org > June 2011

Re: Think before you write Semantic Web crawlers

From: Andreas Harth <andreas@harth.org>
Date: Wed, 22 Jun 2011 11:44:54 +0200
Message-ID: <4E01B996.2010209@harth.org>
To: Christopher Gutteridge <cjg@ecs.soton.ac.uk>
CC: Martin Hepp <martin.hepp@ebusiness-unibw.org>, Daniel Herzig <herzig@kit.edu>, semantic-web@w3.org, public-lod@w3.org 
Hi Christopher,

On 06/22/2011 10:14 AM, Christopher Gutteridge wrote:
> Right now queries to data.southampton.ac.uk (eg.
> http://data.southampton.ac.uk/products-and-services/CupCake.rdf ) are made live,
> but this is not efficient. My colleague, Dave Challis, has prepared a SPARQL
> endpoint which caches results which we can turn on if the load gets too high,
> which should at least mitigate the problem. Very few datasets change in a 24
> hours period.

setting the Expires header and enabling mod_cache in Apache httpd (or adding
a Squid proxy in front of the HTTP server) works quite well in these cases.

Best regards,
Andreas.
Received on Wednesday, 22 June 2011 09:45:29 UTC

This archive was generated by hypermail 2.4.0 : Friday, 17 January 2020 16:21:14 UTC