RE: TDB results for the Berlin SPARQL Benchmark



> -----Original Message-----
> From: Chris Bizer [mailto:chris@bizer.de]
> Sent: 11 August 2008 12:19
> To: Seaborne, Andy; public-sparql-dev@w3.org
> Cc: aschultz@inf.fu-berlin.de
> Subject: Re: TDB results for the Berlin SPARQL Benchmark
>
> Hi Andy,
>
> very promising results. Its great to see another store handling the
> 100M dataset with a good performance.
>
> Andreas will run another experiment with a slightly changed version of
> the benchmark on our machine in about 2 weeks. We will include TDB
> into this run so that we get directly comparable numbers for TDB.
>
> If you have additional ideas for improving the benchmark before the
> second run, please let us know.

I don’t have any major suggestions at the moment.  Joshua suggested publishing the results for checking.  With the parameterization, maybe the query driver could write results to disk with the parameters it used then use some kind of answer template to check the results.  A simpler version might be to publish a separate set of queries with parameter substituted along with the correct results for testing.

The benchmark results for a single store are span 3-4 order of magnitude so there is a question of what is being measured.   Especially for the smaller number, it is presumably the comms and not the SPARQL engine that is the significant cost point and that will change significantly as to whether it is localhost or remote.

> We already added two extra queries to the mix: A CONSTRUCT query and a
> query with unbound predicates.

Great - look forward to it.

        Andy

>
> Cheers
>
> Chris


--------------------------------------------
Hewlett-Packard Limited
Registered Office: Cain Road, Bracknell, Berks RG12 1HN
Registered No: 690597 England

Received on Tuesday, 12 August 2008 15:46:16 UTC