W3C home > Mailing lists > Public > public-lod@w3.org > August 2008

Evaluating Exploration/Semantic Search UIs (was: Re: freebase parallax: user interface for browsing graphs of data)

From: Andreas Harth <andreas.harth@deri.org>
Date: Sun, 17 Aug 2008 22:22:56 +0100
Message-ID: <48A896B0.6020900@deri.org>
To: Michiel Hildebrand <Michiel.Hildebrand@cwi.nl>
CC: David Huynh <dfhuynh@alum.mit.edu>, public-lod@w3.org, semantic-web@w3c.org

Hi Michiel, all,

Michiel Hildebrand wrote:
> Designing user interfaces for "new" ways of exploration is indeed 
> difficult. Evaluating these interfaces is even more difficult.

I agree with you on the difficulty of evaluation new UIs.  Unlinke 
traditional fields
such as IR, there is no corpus or evaluation method available for Semantic
Web data (at least didn't find any).

To help assessing new UIs for Semantic Web data, we've published a 
corpus at [1] (~25m triples, ~5GB), together with a set of real-world 
user tasks.
There's also ratings which can be used for recommendations.

The corpus is created from a number of free datasets about books; we've
consolidated the data and provide data dumps in NQ, RDF, XML, and MARC.
We've choosen books since there is a public domain data available and there
has been work in the digital library area, so there are existing systems to
compare to.

I hope the corpus is useful and provides a starting point for a general 
search and browsing evaluation dataset.  The work is currently in draft 
comments and suggestions for improvement are welcome!


[1] http://sw.deri.org/2008/05/books/

Received on Sunday, 17 August 2008 21:25:05 UTC

This archive was generated by hypermail 2.4.0 : Friday, 17 January 2020 16:20:41 UTC