RE: RDF tools as workhorse

Hi Leigh,

> While Kowari had initially faster loading times than 
> other databases, it quickly ran into memory problems 
> before we'd managed to load our entire dataset.

My project had been dealing with this for some time 
(actually, it's reassuring to hear that somebody else 
experienced it), but finally:

http://prototypo.blogspot.com/2005/09/kowari-memory-leak-found-and-fixed.html

We have done more extensive testing since the patch, and 
the memory situation appears quite acceptable now: (1M 
transactions, including adds/deletes/queries, ending with 
>30M triples)

http://repo1.nsdl.org/cwilper/replay-done.jpg

I'd be interested to hear any details you're willing to
share regarding your Kowari corruption experience (symptoms,
whether it seemed related to the OOM problem) either on 
or off-list.

Cheers,
Chris

-----Original Message-----
From: Leigh Dodds [mailto:leigh@ldodds.com]
Sent: Thu 9/15/2005 7:12 AM
To: Chris Wilper
Cc: Mailing Lists; semantic-web@w3.org
Subject: Re: RDF tools as workhorse
 
Chris Wilper wrote:

> Hi Mark,
> 
> I'd suggest you take a good look at Kowari.  It really excels
> at query performance and scalability compared to anything
> else I've seen in this space.  My own testing has been in the
> 10-20M triple range.  I've heard that Kowari can easily
> handle ten times that, but haven't tested the assertion for
> myself.  

This runs counter to our experience. While Kowari had
initially faster loading times than other databases, it
quickly ran into memory problems before we'd managed to
load our entire dataset.

We also encountered problems with database corruption
and management issues (e.g. taking regular backups).

We ended up using a relationally backed store as we got
the benefits of a stable backend (management tools, replication,
backup, etc) with the RDF model.

Cheers,

L.