[CFP] Final Call for SeMantic AnsweR Type Prediction (SMART) - ISWC 2020 Challenge

ISWC 2020 Challenge - SeMantic AnsweR Type prediction (SMART) task.

    in conjunction with ISWC 2020 [https://iswc2020.semanticweb.org/]

*********************************************************************************

Challenge website: https://smart-task.github.io/

Conference date and location: 2-6 November 2020, (Online - Virtual)

*Submission deadline: September 21, 2020*

*********************************************************************************

UPDATES:

* Extended deadlines. Please refer to the website for the updated timeline.

* Test data and evaluation scripts are now available, please refer to the
website for details.

* A Slack channel is available for any questions/discussions - please send
an email to get an invitation.

Brief Background

SMART task is focused on answer type prediction. Question or answer type
classification plays a key role in question answering. The questions can be
generally classified based on Wh-terms (Who, What, When, Where, Which,
Whom, Whose, Why). Similarly, the answer type classification is determining
the type of the expected answer based on the query. Such answer type
classifications in literature have been performed as a short-text
classification task using a set of coarse-grained types, for instance,
either 6 or 50 types within the TREC QA task. A granular answer type
classification is possible with popular Semantic Web ontologies such as
DBpedia (~760 classes) and Wikidata (~50K classes).

Task Description

In this challenge, given a question in natural language, the task is to
predict the type of the answer using a set of candidates from a target
ontology. e.g.,

Who is the heaviest player of the Chicago Bulls? -> dbo:BasketballPlayer

How many employees does IBM have? -> number

When did Margaret Mead marry Gregory Bateson? -> date

Datasets

We have created two datasets; one using the DBpedia ontology and the other
using the Wikidata ontology each containing 21,964 questions and 22,822
questions respectively. Systems can participate using only one dataset or
both.

Submission Details

Participants are requested to submit the system output for the test data.
The format is as same as the training data. In addition, the participants
are requested to submit a system description that will be included in a
joint ISWC challenge proceedings volume in CEUR. System descriptions must
be in English either in PDF or HTML, formatted in the style of LNCS, and no
longer than 8 pages. Submissions can be sent via email. The accepted
systems will get the opportunity to show their results during the ISWC 2020
poster and demo session.

Organizers

Mohnish Dubey, University of Bonn / Fraunhofer IAIS Dresden

Alfio Gliozzo, IBM Research AI

Jens Lehmann, University of Bonn / Fraunhofer IAIS Dresden

Nandana Mihindukulasooriya, IBM Research AI

Axel-Cyrille Ngonga Ngomo, Universität Paderborn

Ricardo Usbeck, Fraunhofer IAIS Dresden