[Call for submissions] Semantic Reasoning Evaluation Challenge (SemREC) at ISWC 2023

Dear All,

TL;DR

************************

This is the first call for participation for the third edition of the Semantic
Reasoning Evaluation Challenge (SemREC) <https://semrec.github.io/> collocated
with the 22nd International Semantic Web Conference
<https://iswc2023.semanticweb.org/>.

   1. *Ontologies - Submit a real-world ontology that is a challenge in
   terms of the reasoning time or memory consumed during reasoning. We expect
   a detailed description of the ontology along with the analysis of the
   reasoning performance, the workarounds if any, that were used to make the
   ontology less challenging (for example, dropping a few axioms, redesigning
   the ontology, etc.), and the (potential) applications in which the ontology
   could be used. We will be evaluating the submitted ontologies based on the
   time consumed for a reasoning task, such as classification, and the memory
   consumed during reasoning.*
   2. Systems - Submit an ontology/RDFS reasoner that uses neural-symbolic
   techniques for reasoning and optimization.


Deadline - 15 August 2023, 23:59 (AOE)

If you have any questions, please contact me (gunjans@iiitd.ac.in).

Further details are on the website: https://semrec.github.io/.

************************

Longer version

*************************

Despite the development of several ontology reasoning optimizations, the
traditional methods either do not scale well or only cover a subset of OWL
2 language constructs. As an alternative, neuro-symbolic approaches are
gaining significant attention. However, the existing methods can not deal
with very expressive ontology languages. Other than that, some SPARQL query
engines also support reasoning, but their performance also is still
limited. To find and improve these performance bottlenecks of the
reasoners, we ideally need several real-world ontologies that span the
broad spectrum in terms of their size and expressivity. However, that is
often not the case. One of the potential reasons for ontology developers to
not build ontologies that vary in terms of size and expressivity is the
performance bottleneck of the reasoners. SemREC aims to deal with this
chicken and egg problem.

The second edition of this challenge includes the following tasks-

   -

   Task-1 - Ontologies. Submit a real-world ontology that is a challenge in
   terms of the reasoning time or memory consumed during reasoning. We expect
   a detailed description of the ontology along with the analysis of the
   reasoning performance, the workarounds if any, that were used to make the
   ontology less challenging (for example, dropping a few axioms, redesigning
   the ontology, etc.), and the (potential) applications in which the ontology
   could be used. We will be evaluating the submitted ontologies based on the
   time consumed for a reasoning task, such as classification, and the memory
   consumed during reasoning.
   -

   Task-2 - Systems
   -

      Ontology/RDFS Reasoners. Submit an ontology/RDFS reasoner that uses
      neural-symbolic techniques for reasoning and optimization. In terms of
      technique used, the submissions could fall under any of the below (or
      related) categories.
      1.

         Using learning-based techniques for performance optimization of
         traditional reasoning algorithms.
         2.

         Inductive reasoning techniques based on a subsymbolic
         representation of entities and relations learned through the
maximization
         of an objective function over valid triples.
         3.

         Techniques that can learn the deductive reasoning aspect using the
         ontology axioms.
         4.

         Neural Multi-hop reasoners to deal with reasoning where multi-hop
         inference is required.

Based on precision and recall, we will evaluate the submitted systems on
the test datasets for scalability (performance evaluation on large and
expressive ontologies) and transfer capabilities (ability to reason over
ontologies from different domains). We expect a detailed description of the
system, including an evaluation of the system on the provided datasets.

   -

   SPARQL query engines that support entailment regimes such as RDF, RDFS,
   or OWL 2. We expect a detailed description of the system, including an
   evaluation of the system on the provided datasets.


Submission Details

Participants are requested to make a manuscript submission describing their
entry.

   -

   For Task 1, we expect a detailed description of the ontology along with
   the analysis of the reasoning performance, and the workarounds, if any,
   that were used to make the ontology less challenging (for example, dropping
   a few axioms, redesigning the ontology, etc.), and the (potential)
   applications in which the ontology could be used.
   -

   For Task 2, we expect a detailed description of the system, including
   evaluating the system on the provided datasets. We will further evaluate
   the submitted systems on a subset of the test datasets for scalability
   (performance evaluation on large and expressive ontologies) and transfer
   capabilities (ability to reason over ontologies from different domains).


The submissions can be either in the form of short papers of length 5 pages
or long papers of length 10-12 pages. All the submissions must be in
English and follow the 1-column CEUR-ART style (overleaf template)
<https://www.overleaf.com/latex/templates/ceurart-template-for-submissions-to-semrec-challenge/qktzxsbyhsdp>.
The proceedings will be published as a volume of CEUR-WS
<http://ceur-ws.org/>. Submissions should be made in the form of a pdf
document on EasyChair.

Website: https://semrec.github.io/

Organizers

Gunjan Singh, IIIT-Delhi, India.

Raghava Mutharaju, IIIT-Delhi, India.

Pavan Kapanipathi, IBM T.J. Watson Research Center, USA.

Best regards,

Gunjan

--
Thanks and Regards

Gunjan Singh
Ph. D Student, KRaCR Lab <https://kracr.iiitd.edu.in/>, IIIT-Delhi
<https://cse.iiitd.ac.in/>, India
IBM Research Fellow