[CFP] 1st Call for Participation - CLEF 2024 SimpleText Task4: SOTA? "Tracking the State-of-the-Art in Scholarly Publications"

 [apologies if you receive multiple copies of this call]

Dear colleagues and friends,

*We are pleased to release the 1st Call for Participation - CLEF 2024
SimpleText Task4: SOTA?*

*Overview:* SOTA? is introduced as Task 4 in the SimpleText track of CLEF
2024. The goal of the SOTA? shared task is to develop systems which given
the full text of an AI paper, are capable of recognizing whether an
incoming AI paper indeed reports model scores on benchmark datasets, and if
so, to extract all pertinent (Task, Dataset, Metric, Score) quadruples
presented within the paper.

More info on the task website:
https://sites.google.com/view/simpletext-sota/home

SOTA? will be divided into two evaluation phases:

- Evaluation Phase 1: Few-shot Testing;
- Evaluation Phase 2: Zero-shot Testing

*To participate in SOTA? i.e. SimpleText Task 4 @ CLEF 2024, please
register your team*:

1. CLEF 2024 official registration page
https://clef2024.imag.fr/index.php?page=Pages/registration.html
2. Codalab competition site:
https://codalab.lisn.upsaclay.fr/competitions/16616

Note, SOTA? is organized as a new task this year under the "SimpleText -
Improving Access to Scientific Texts for Everyone" initiative
https://simpletext-project.com/. Please take a look at the other 3 tasks,
i.e. Task 1, 2, and 3, offered by SimpleText and select one or more of
those task options too if you are interested. Note that there is no
interdependence of the dataset between "Task 4 - SOTA?" and the other three
tasks of SimpleText.

*Dates*

Training and validation datasets available: Feb 1, 2024
Test data available/Evaluation starts: April 23, 2024
Evaluation ends: May 3, 2024
Participant paper submissions due: May 31, 2024
Notification to authors: June 24, 2024
Camera ready due: July 8, 2024
CLEF 2024 Workshop, Grenoble, France: 9-12 September 2024

*Task Organizers*

Jennifer D’Souza (TIB Leibniz Information Centre for Science and Technology
- Germany)
Salomon Kabongo (L3S Research Center, Germany)
Hamed Babaei Giglou (TIB Leibniz Information Centre for Science and
Technology - Germany)
Yue Zhang (Berlin Technical University, Germany)
Sören Auer (TIB Leibniz Information Centre for Science and Technology -
Germany)

*We look forward to having you on board!*

*Contact:* sota.task [at] gmail.com