Evaluating Exploration/Semantic Search UIs (was: Re: freebase parallax: user interface for browsing graphs of data)

Hi Michiel, all,

Michiel Hildebrand wrote:
> Designing user interfaces for "new" ways of exploration is indeed 
> difficult. Evaluating these interfaces is even more difficult.

I agree with you on the difficulty of evaluation new UIs.  Unlinke 
traditional fields
such as IR, there is no corpus or evaluation method available for Semantic
Web data (at least didn't find any).

To help assessing new UIs for Semantic Web data, we've published a 
medium-size
corpus at [1] (~25m triples, ~5GB), together with a set of real-world 
user tasks.
There's also ratings which can be used for recommendations.

The corpus is created from a number of free datasets about books; we've
consolidated the data and provide data dumps in NQ, RDF, XML, and MARC.
We've choosen books since there is a public domain data available and there
has been work in the digital library area, so there are existing systems to
compare to.

I hope the corpus is useful and provides a starting point for a general 
semantic
search and browsing evaluation dataset.  The work is currently in draft 
quality;
comments and suggestions for improvement are welcome!

Regards,
Andreas.

[1] http://sw.deri.org/2008/05/books/

-- 
http://swse.deri.org/

Received on Sunday, 17 August 2008 21:25:05 UTC