Re: TDB results for the Berlin SPARQL Benchmark

Hi Andy,

very promising results. Its great to see another store handling the 
100M dataset with a good performance.

Andreas will run another experiment with a slightly changed version of 
the benchmark on our machine in about 2 weeks. We will include TDB 
into this run so that we get directly comparable numbers for TDB.

If you have additional ideas for improving the benchmark before the 
second run, please let us know.
We already added two extra queries to the mix: A CONSTRUCT query and a 
query with unbound predicates.

Cheers

Chris


> This report is for TDB running the Berlin SPARQL Benchmark [1]
> TDB is a persistent subsystem for Jena.
>
> Thanks to Andreas Schultz who provided the configuration files and 
> updated tools necessary to run the benchmark on the downloaded 
> datasets.
>
> Date: 2008-08
>
> == Results
>
> = Load
> Using TDB's bulk loader: "tdbloader"
> Cold load into an empty database.
>
>          50K      250K        1M         5M      25M       100M
> Time/s   5.56     15.25     50.46     256.65   1258.07    5370.09
> TPS     9,020    16,427    19,824     19,483    19,872     18,621
>
> = Query
>            50K         250K         1M          5M         25M 
> 100M
> Query 1   0.007711  0.036216    0.066769    0.137431    0.442776 
> 1.463148
> Query 2   0.032370  0.046859    0.047317    0.047442    0.046882 
> 0.056338
> Query 3   0.008156  0.021385    0.058452    0.141895    0.410103 
> 1.429356
> Query 4   0.011558  0.019491    0.067215    0.220680    0.654449 
> 2.571348
> Query 5   0.069205  0.357140    0.973735    3.624503   21.729141 
> 84.927111
> Query 6   0.005759  0.043215    0.049023    0.062252    0.524726 
> 11.309403
> Query 7   0.037831  0.045588    0.197200    0.436487    0.711938 
> 0.771399
> Query 8   0.027343  0.060075    0.235520    0.480224    0.798117 
> 0.834888
> Query 9   0.006449  0.006994    0.009983    0.021610    0.032256 
> 0.041381
> Query 10  0.004890  0.014506    0.060580    0.103583    0.187552 
> 0.197450
>
> == Software
>
> Jena 2.5.6  <http://jena.sf.net/>
> ARQ 2.4     <http://jena.sf.net/ARQ>
> TDB 0.5     <http://jena.sf.net/TDB>
> Joseki 3.2  <http://www.joseki.org>
> Java        1.6.0_02
>            Sun Java HotSpot(TM) 64-Bit Server VM (build 
> 1.6.0_02-b05, mixed mode)
> OS: RHEL5
>
> TDB configuration:
>  None (TDB has no configuration options)
>
> == Hardware
>
> CPU: 4 CPU: AMD Opteron(tm) Processor 280 @1.8GHz
> Memory:   10Gbytes
>
> Note: this used non-local disk, using a shared disk array in a data 
> center.  This provides a more realistic set up of enterprise use. It 
> does make it difficult to draw deep conclusions when comparing with 
> different setups using local disk.
>
> Disk: HP EVA8000 running firmware xcsp-6100.
>
> The LUNs are configured as VRAID5, with read cache enabled and write 
> cache in write-back mode.  Two Qlogic fibre-channel ports (2GB 
> speed) connected to an MDS 9140 edge switch each, then an MDS9509 
> core switch and finally the EVA.
>
>    Andy
>
> [1] http://www4.wiwiss.fu-berlin.de/bizer/BerlinSPARQLBenchmark/
>
> --------------------------------------------
>  Hewlett-Packard Limited
>  Registered Office: Cain Road, Bracknell, Berks RG12 1HN
>  Registered No: 690597 England
>

Received on Monday, 11 August 2008 11:19:51 UTC