W3C home > Mailing lists > Public > semantic-web@w3.org > February 2010

Re: SPARQL & memory use

From: David Booth <david@dbooth.org>
Date: Mon, 08 Feb 2010 09:19:00 -0500
To: Danny Ayers <danny.ayers@gmail.com>
Cc: Semantic Web <semantic-web@w3.org>
Message-ID: <1265638740.4169.9558.camel@dbooth-laptop>
FYI, last I recall, I think Jena used roughly 1KB/triple total memory,
including all indexes, etc.  If someone has more current or precise
numbers, please update/correct me.

David Booth


On Sun, 2010-02-07 at 21:30 +0100, Danny Ayers wrote:
> A little while ago I was convinced I should learn Scala (thanks Reto)
> - not there yet by any means, but noticed a few things along the way.
> If you use Jena, try Scala - you have the same lib stuff available but
> in a syntax that's a lot friendlier than Java.
> 
> Already I digress.
> 
> So my server, on which I run a blog, is falling over on a daily basis
> with 1GB immediate mem (+2GB swap) available (and a lousy admin).
> Today, like a fool I coded a SPARQL endpoint for it.
> 
> Here's the rub I want to rub against you - to get even moderately
> general queries I had to say -Xmx2048m -Xmn1024m, which damn near
> drove my local laptop (with 4GB available) into the ground.
> 
> Which is where I could use your expertise.
> 
> Because I was only working with only a tiny model (about 40MB, most of
> it literals) I thought I could keep that in mem, query away. D'oh!
> 
> Would using an e.g. MySQL-backed model help here? Or something else?
> 
> What strategies (and code?) do we have to detect a memory-munching query?
> 
> Or should I simply go on the game to raise funds for more slicehost memory?
> 
> The thing is is here:
> http://dannyayers.com
> 
> (already set to clear the decks every hour)
> 
> Source is near here:
> http://hyperdata.org/wiki/wiki/Gradino
> 
> Cheers,
> Danny.
> 


-- 
David Booth, Ph.D.
Cleveland Clinic (contractor)

Opinions expressed herein are those of the author and do not necessarily
reflect those of Cleveland Clinic.
Received on Monday, 8 February 2010 14:19:28 GMT

This archive was generated by hypermail 2.3.1 : Tuesday, 26 March 2013 21:45:34 GMT