Re: RDF benchmarks

On Sun, 21 Apr 2002, Geoff Chappell wrote:

> Does anyone know of any benchmarks for RDF query systems and/or
> repositories?  I'd like to have some standard measure of (at least):
> (a) triple insertion/deletion time, and (b) query performance against
> a sizeable kb (with and without inference).  I'd guess that the
> systems out there today vary enough in their capabilities that a
> single benchmark might be difficult to come up with, so I imagine some
> categorization of benchmarks and results would be necessary.

Amen to that. We are currently in the process of doing some performance
analysis on the Sesame system, but without a standard benchmark it is
very hard to compare performance with other systems.  Basically, our
approach sofar has been to take a "well-known" RDF dataset (like the CIA
Factbook, or parts of the UNSPSC ontology), and do performance measuring
on that. But this is of course ad hoc.

I do know that there is an initiative going on in the Ontology Tools SIG
of the OntoWeb network (see http://www.ontoweb.org/) to come up with a
set of benchmarks, but I am unaware of how far this idea has gone beyond
the "coffee table".


Regards,

Jeen
-- 
Jeen Broekstra                                     Vrije Universiteit
jbroeks@cs.vu.nl              Dept. of Mathematics & Computer Science
                                 de Boelelaan 1081, 1081 HV Amsterdam
                                                      The Netherlands