RFC: Berlin SPARQL Benchmark

Hi all,

SPARQL query language and the SPARQL protocol are implemented by a 
growing number of storage systems and are used within enterprise and 
open web settings. As SPARQL is taken up by the community there is a 
growing need for benchmarks to compare the performance of storage 
systems that expose SPARQL endpoints via the SPARQL protocol.

We have been working over the last week on such a benchmark called the 
Berlin SPARQL Benchmark (BSBM). The benchmark is built around an 
e-commerce use case in which a set of products is offered by different 
vendors and consumers have posted reviews about products. The 
benchmark query mix illustrates the search and navigation pattern of a 
consumer looking for a product.

We have also run the initial version of the benchmark against Sesame, 
Virtuoso, Jena SDB  and against D2R Server, a relational 
database-to-RDF wrapper. The stores were benchmarked with datasets 
ranging from 50,000 triples to 100,000,000 triples.

Our current benchmark spec as well as the results of our initial 
experiments are found at:

http://www4.wiwiss.fu-berlin.de/bizer/BerlinSPARQLBenchmark/

In order to improve the Berlin SPARQL benchmark, it would be great to 
get feedback from the community on the benchmark specification. So if 
you think we have missed something essential or if you have ideas for 
further improvements, please let us know by replying to this mail or 
by contacting us directly.

Also note that there is ongoing work on a second SPARQL benchmark: The 
SP2B SPARQL Performance Benchmark 
(http://dbis.informatik.uni-freiburg.de/index.php?project=SP2B). 
Compared with the BSBM benchmark, SP2B focuses more on testing 
different storage layouts and RDF data processing approaches while we 
try to be strictly use case driven.

We are looking forward to your feedback :-)

Cheers

Chris and Andreas

--
Chris Bizer
Freie Universität Berlin
Phone: +49 30 838 55509
Mail: chris@bizer.de
Web: www.bizer.de

Received on Wednesday, 30 July 2008 16:21:14 UTC