1st Challenge on Question Answering over Linked Data (QALD-1)

The challenge on Question Answering over Linked Data is on!.

Please find below important information. We would be happy if you  
participate.


[Apologies for cross-posting]

====================================


1st Challenge on Question Answering over Linked Data (QALD-1)

	http://www.sc.cit-ec.uni-bielefeld.de/qald-1

		   collocated with the corresponding workshop at the

	   Extended Semantic Web Conference (ESWC)

                     http://www.eswc2011.org/


* Motivation *

While more and more semantic data is published on the Web, in
particular following the Linked Data principles, the question of how
typical Web users can access this body of knowledge through an
intuitive and easy-to-use interface that hides the complexity of the
Semantic Web standards becomes of crucial importance. Since users
prefer to express their information need in natural language, one of
the main challenges lies in translating the user's information needs
into a form such that they can be automatically processed using
standard Semantic Web query processing and inferencing techniques. In
recent years, there have been important advances in semantic search
and question answering over RDF data, and in parallel there has been
substantial progress on question answering from textual data as well
as in the area of natural language interfaces to databases. This
shared task and the associated workshop aim at bringing together
researchers from these communities that accept the challenge of
scaling question answering approaches to the growing amount of
heterogeneous and distributed Linked Data. Our long-term goal is to
understand how we can develop QA approaches that deal with the fact
that i) the amount of RDF data available on the Web is huge, ii) that
this data is distributed and iii) that it is heterogeneous with
respect to the vocabularies or schemas used.

* Shared Task *

Question answering systems of all kinds are invited to participate in
the shared task of processing natural language queries and retrieving
relevant answers from a given RDF dataset, thereby providing an
in-depth view of the strength, capabilities and shortcomings of
existing systems. Please note that although some of the questions are
quite complex, nonetheless we would like to encourage everybody to
participate in the challenge even if they can only successfully
process a subset of the questions. Although the competition is
tailored towards question answering systems based on natural
language, we strongly encourage other relevant systems and methods
that can benefit from the evaluation datasets to also report their
results.

* Datasets *

We provide two datasets: DBpedia 3.6 and MusicBrainz. They can either
be downloaded or accessed via a SPARQL endpoint. In addition, we
provide 50 training questions for each dataset, annotated with
corresponding SPARQL queries and answers. Later, during the test
phase, participating systems will be evaluated with respect to
precision and recall on a set of 50 similar questions.

* Submission *

Submission of results will be possible via an online form, available
at the following site http://www.sc.cit-ec.uni-bielefeld.de/qald-1
(from February 7th on). Submission of results on training data will
be allowed at any time. Participants will receive the results on the
training data for every submission. Submission of results on test
data will be possible starting from April 1st and close on April
10th. Participants can submit as many runs as they want but will not
receive any feedback.


* Schedule *

Release of QA training dataset and instructions: Feb 3
Release of QA testset: March 28
Submission of results by participants: April 1
Close of result submission on test data: April 10

* More information *

For detailed information as well as links to the datasets and
training questions, please check the workshop website:
http://www.sc.cit-ec.uni-bielefeld.de/qald-1

If you want to be regularly updated with information about the task,
there is a QALD-1 mailing list, to which you can subscribe at the
following location:

https://lists.techfak.uni-bielefeld.de/cit-ec/mailman/listinfo/qald-1

All the best!


-- 
The Open University is incorporated by Royal Charter (RC 000391), an exempt charity in England & Wales and a charity registered in Scotland (SC 038302).

Received on Friday, 4 February 2011 16:14:04 UTC