W3C home > Mailing lists > Public > www-webont-wg@w3.org > June 2005

Ontology Alignment Evaluation 2005

From: Jerome Euzenat <Jerome.Euzenat@inrialpes.fr>
Date: Tue, 14 Jun 2005 16:16:48 +0200
Message-Id: <a06001a0dbed488746b92@[]>
To: (Recipient list suppressed)

                Ontology Alignment Evaluation Initiative
                            2005 Campaign
                        Call for participation



The increasing number of methods available for schema 
matching/ontology integration suggests the need to establish a 
consensus for evaluation of these methods. There is now a coordinated 
international initiative to forge this consensus. After the two 
events organized in 2004 (namely, the Information Interpretation and 
Integration Conference (I3CON) and the EON Ontology Alignment 
Contest), this year we organize one unique evaluation of which the 
outcome will be presented at the Workshop on Integrating Ontologies 
held in conjunction with K-CAP 2005 at Banff (Canada) on October 2, 

The goal of the ontology alignment initiative is to provide ground to 
discuss alignment and matching algorithms and system performances and 
to help as much as possible their improvements.

Campaign description

This year's campaign will consist of three parts: it will feature two 
real world blind tests (anatomy and directory) and a systematic 
benchmark test suite. By blind tests it is meant the result expected 
from the test is not known in advance. The evaluation organisers 
provide the participants with the pairs of ontologies to align as 
well as (in the case of the systematic benchmark suite only) expected 
results. The ontologies are described in OWL-DL and serialized in the 
RDF/XML format. The expected alignments are provided in a standard 
format expressed in RDF/XML and described in 

The anatomy real world case will have to be tested blindly. It covers 
the domain of body anatomy and will consist of two ontologies with an 
approximate size of several 10k classes and several dozen of 

The directory real world case will have to be tested blindly. It 
consist of alignming web sites directory (like open directory or 

Like for last year's EON contest, a systematic benchmark series has 
been produced. The goal of this benchmark series is to identify the 
areas in which each alignment algorithm is strong and weak. The test 
is based on one particular ontology dedicated to the very narrow 
domain of bibliography and a number of alternative ontologies of the 
same domain for which alignments are provided.

 From the results of the experiments the participants are expected to 
provide the organisers with a paper to be published in the workshop 

The results from both, the participants and the organizers, will be 
presented at the Workshop on Integrating Ontologies at K-CAP 2005 
taking place at Banff (Canada) on October, 2nd 2005. We hope to see 
you there.


First publication of test cases: June 10th, 2005.
Comments due: anytime before July 1st, 2005
Final publication of test cases: July 4th, 2005.
Preliminary results due: August 15th, 2005.
Camera ready copies: September 2nd, 2005.
Workshop : October 2nd, 2005.

Steering Committee

Benjamin Ashpole (Lockheed Martin Advanced Technology Lab.)
Marc Ehrig (University of Karlsruhe)
Jérôme Euzenat (INRIA Rhône-Alpes)
Lewis Hart (Applied Minds)
Todd Hughes (Lockheed Martin Advanced Technology Labs)
Natasha Noy (Stanford University)
Heiner Stuckenschmidt (Vrije Universiteit Amsterdam)
Petko Valtchev (Université de Montréal, DIRO)

Contact addresses are heiner (à) cs . vu . nl and Jerome . Euzenat 
(à) inrialpes . fr
Received on Tuesday, 14 June 2005 14:33:56 UTC

This archive was generated by hypermail 2.3.1 : Tuesday, 6 January 2015 21:56:58 UTC