- From: David Schmidt <david.schmidt@uni-bielefeld.de>
- Date: Fri, 17 Apr 2026 13:59:12 +0200
- To: <semantic-web@w3.org>
[Apologies for cross-postings] Dear colleagues, We are pleased to announce WikiKGQA: Wiki-Based Knowledge Graph Question Answering Challenge co-located with the 25th International Semantic Web Conference (https://iswc2026.semanticweb.org/), Bari, Italy. Challenge website: https://wikikgqa.org The competition takes place on the Codabench platform: https://www.codabench.org/competitions/15358/ Important links and dates: - Training data + challenge platform: https://www.codabench.org/competitions/15358/ - Challenge solution submission: June 19, 2026 (11:59pm, AoE) - Challenge results: July 10, 2026 - Paper submissions: July 17, 2026 (11:59pm, AoE) - Paper acceptance notification: August 21, 2026 - Workshop days: October 25 OR 26, 2026 Motivation: KGQA has a long tradition in the Semantic Web community, particularly under the Question Answering over Linked Data (QALD) moniker and challenge series. With the rise of LLMs, the performance of state-of-the-art systems has improved dramatically. However, benchmark datasets in the space exhibit well-known limitations, primarily due to the manual cost of creating parallel corpora of natural language questions and structured queries. To address these issues, QAWiki (https://qawiki.org) proposes a collaboratively-edited community resource of hand-crafted question-query pairs, focusing currently on English & Spanish questions answered over Wikidata's SPARQL query service. The motivation is two-fold: (1) to evaluate KGQA systems over a hand-crafted collection of multilingual question-query pairs over Wikidata, and (2) to build a community centered around QAWiki, to expand and improve it, and thus expand and improve the resources available for training and benchmarking of KGQA systems. Task Description: Knowledge Graph Question Answering (KGQA) offers a promising alternative to end-to-end QA systems by obtaining truthful answers from trusted and well-maintained knowledge sources, such as Wikidata, without losing the flexibility of natural language input. Nevertheless, the creation of large high-quality KGQA benchmarks remains a challenge, with many benchmarks being small, containing errors or repetitive questions that are rarely fixed. Therefore, building on QAWiki, this challenge aims to make the creation of KGQA benchmarks a collaborative effort, just like Wikidata itself. For evaluation, a private test dataset will be created, which would be integrated with QAWiki after the challenge. Over the years, we aim to build an evolving high-quality KGQA dataset of increasing size. English and Spanish queries will be ranked separately, each participant can choose to participate only for one or for both languages. The task itself consists of two sub-tasks for each language: 1. KGQA using only the input question 2. KGQA using the input question together with the metadata QAWiki provides, i.e., entity and property annotations In both sub-tasks, the goal is to generate an answer set for that question based on the version of Wikidata accessible via the WikiKGQA SPARQL endpoint or using the available Wikidata dump (more details at the Codabench competition page). As an evaluation metric, Macro QALD F1 scores will be used to evaluate the result sets. The full training data is available under https://www.codabench.org/competitions/15358/ as an extended version of QALD_JSON (https://www.nliwod.org/challenge#task-1-multilingual-question-answering-over-knowledge-graphs). Submission Guidelines: We invite submissions of English challenge papers (up to 8 pages excluding references). All submissions should be formatted in the CEUR layout https://www.overleaf.com/latex/templates/template-for-submissions-to-ceur-workshop-proceedings-ceur-ws-dot-org/wqyfdgftmcfw A prerequisite for submitting a challenge paper is to submit challenge solutions of your approach to at least one of the tracks on Codabench: https://www.codabench.org/competitions/1535. Additionally, each participant/team has to let the organizers know their team name such that Codabench submissions can be linked to paper submissions later. Submission link: https://easychair.org/conferences/?conf=wikikgqa2026 Codabench: https://www.codabench.org/competitions/15358/ Google Groups list: https://groups.google.com/g/wikikgqa-challenge-list/ The central communication platform for the challenge (aside from the website) is the corresponding Google Groups list: https://groups.google.com/g/wikikgqa-challenge-list/ For private concerns or questions, please contact David Maria Schmidt (david.schmidt@uni-bielefeld.de) or any other organizer directly. Follow us on social media! LinkedIn: https://www.linkedin.com/company/wikikgqa/ Bluesky: https://bsky.app/profile/wikikgqa.bsky.social Mastodon: https://mastodon.social/@wikikgqa Workshop Organizers: - Shakeeb Arzoo, CRISIL Ltd., S&P Global Ratings - Debayan Banerjee, Leuphana University Lüneburg, Germany - Philipp Cimiano, CITEC, Bielefeld University, Germany - Aidan Hogan, DCC, Universidad de Chile - Alberto Moya Loustaunau, DCC, Universidad de Chile - David Maria Schmidt, CITEC, Bielefeld University, Germany - Ricardo Usbeck, Leuphana University Lüneburg, Germany -- David Maria Schmidt Cognitive Interaction Technology Center (CITEC) Universität Bielefeld Inspiration 1 33619 Bielefeld Germany Mail: david.schmidt@uni-bielefeld.de Web: https://davidmariaschmidt.de/
Received on Friday, 17 April 2026 20:36:38 UTC