GWAP: Evaluating Summaries

Dear all,

We would like to invite you to try the latest creation of 
WhoKnows?Movies! [1]

WhoKnows?Movies! is a GWAP (game with a purpose) that asks quiz 
questions about famous movies. The data comes from Freebase. We also 
have a high score list which we will keep online for eternity (you can 
mention it in your CV)!

Game with a purpose - "but what actually is the purpose?":
Summarizing entities individually in accordance to the relevance of 
specific property-value pairs has gained focus of recent efforts by 
research (e.g. [2]) and also industry (e.g. [3]). Unfortunately, such 
summaries are hard to evaluate due to the lack of a gold standard. In 
our paper - accepted for this year's ISWC Experiments and Evaluation 
track - we address this issue with a GWAP. We have published a first 
(anonymized) excerpt of the data gathered via the game at [4]. However, 
such a dataset is never complete and we want to provide an enhanced 
version for the camera ready version of the paper. More information can 
be found on [5].

Kind Regards,
Andreas

[1] http://j.mp/WhoKnowsMovies
[2] Gong Cheng, Thanh Tran and Yuzhong Qu. "RELIN: Relatedness and 
Informativeness-based Centrality for Entity Summarization" 
(http://iswc2011.semanticweb.org/fileadmin/iswc/Papers/Research_Paper/09/70310113.pdf)
[3] 
http://googleblog.blogspot.com/2012/05/introducing-knowledge-graph-things-not.html 
("2. Get the best summary")
[4] http://www.yovisto.com/labs/iswc2012/
[5] 
http://moresemantic.blogspot.co.at/2012/08/who-knows-movies-2nd-round.html


-- 
Andreas Thalhammer
PhD Student
Semantic Technology Institute
University of Innsbruck
http://www.sti2.at/

phone: +43 (0) 512507 6454
email:andreas.thalhammer@sti2.at

Received on Wednesday, 22 August 2012 11:40:10 UTC