Re: GWAP: Evaluating Summaries

On 8/22/12 7:39 AM, Andreas Thalhammer wrote:
> Dear all,
>
> We would like to invite you to try the latest creation of 
> WhoKnows?Movies! [1]
>
> WhoKnows?Movies! is a GWAP (game with a purpose) that asks quiz 
> questions about famous movies. The data comes from Freebase. We also 
> have a high score list which we will keep online for eternity (you can 
> mention it in your CV)!
>
> Game with a purpose - "but what actually is the purpose?":
> Summarizing entities individually in accordance to the relevance of 
> specific property-value pairs has gained focus of recent efforts by 
> research (e.g. [2]) and also industry (e.g. [3]). Unfortunately, such 
> summaries are hard to evaluate due to the lack of a gold standard. In 
> our paper - accepted for this year's ISWC Experiments and Evaluation 
> track - we address this issue with a GWAP. We have published a first 
> (anonymized) excerpt of the data gathered via the game at [4]. 
> However, such a dataset is never complete and we want to provide an 
> enhanced version for the camera ready version of the paper. More 
> information can be found on [5].
>
> Kind Regards,
> Andreas
>
> [1] http://j.mp/WhoKnowsMovies
> [2] Gong Cheng, Thanh Tran and Yuzhong Qu. "RELIN: Relatedness and 
> Informativeness-based Centrality for Entity Summarization" 
> (http://iswc2011.semanticweb.org/fileadmin/iswc/Papers/Research_Paper/09/70310113.pdf)
> [3] 
> http://googleblog.blogspot.com/2012/05/introducing-knowledge-graph-things-not.html 
> ("2. Get the best summary")
> [4] http://www.yovisto.com/labs/iswc2012/
> [5] 
> http://moresemantic.blogspot.co.at/2012/08/who-knows-movies-2nd-round.html
>
>
Andreas,

Great stuff!

I am assuming that the buy-product of this effort is LOD data cleansing? 
If so, is there an RDF based Linked Data archive (a linkset) that can 
then be uploaded to the LOD cloud etc? For instance, is the archive in 
question part of the resource at: 
http://www.yovisto.com/labs/iswc2012/google_summaries.tar.gz ?

-- 

Regards,

Kingsley Idehen	
Founder & CEO
OpenLink Software
Company Web: http://www.openlinksw.com
Personal Weblog: http://www.openlinksw.com/blog/~kidehen
Twitter/Identi.ca handle: @kidehen
Google+ Profile: https://plus.google.com/112399767740508618350/about
LinkedIn Profile: http://www.linkedin.com/in/kidehen