Re: SPARQL & memory use

FYI, last I recall, I think Jena used roughly 1KB/triple total memory,
including all indexes, etc.  If someone has more current or precise
numbers, please update/correct me.

David Booth


On Sun, 2010-02-07 at 21:30 +0100, Danny Ayers wrote:
> A little while ago I was convinced I should learn Scala (thanks Reto)
> - not there yet by any means, but noticed a few things along the way.
> If you use Jena, try Scala - you have the same lib stuff available but
> in a syntax that's a lot friendlier than Java.
> 
> Already I digress.
> 
> So my server, on which I run a blog, is falling over on a daily basis
> with 1GB immediate mem (+2GB swap) available (and a lousy admin).
> Today, like a fool I coded a SPARQL endpoint for it.
> 
> Here's the rub I want to rub against you - to get even moderately
> general queries I had to say -Xmx2048m -Xmn1024m, which damn near
> drove my local laptop (with 4GB available) into the ground.
> 
> Which is where I could use your expertise.
> 
> Because I was only working with only a tiny model (about 40MB, most of
> it literals) I thought I could keep that in mem, query away. D'oh!
> 
> Would using an e.g. MySQL-backed model help here? Or something else?
> 
> What strategies (and code?) do we have to detect a memory-munching query?
> 
> Or should I simply go on the game to raise funds for more slicehost memory?
> 
> The thing is is here:
> http://dannyayers.com
> 
> (already set to clear the decks every hour)
> 
> Source is near here:
> http://hyperdata.org/wiki/wiki/Gradino
> 
> Cheers,
> Danny.
> 


-- 
David Booth, Ph.D.
Cleveland Clinic (contractor)

Opinions expressed herein are those of the author and do not necessarily
reflect those of Cleveland Clinic.

Received on Monday, 8 February 2010 14:19:28 UTC