CFP: 6th International Workshop on Evaluation of Ontology-based tools and the Semantic Web Challenge (EON-SWSC2008)

***********************************************************************
*                                                                     *
*                             Call for papers                         *
*                                                                     *
*            Sixth International Workshop on Evaluation of            *
*              Ontology-based tools and the Semantic Web              *
*                   Service Challenge (EON-SWSC2008)                  *
*       at the 5th European Semantic Web Conference (ESWC2008)        *
*                Sheraton La Caleta, Tenerife, Spain                  *
*                       June 1st-2nd, 2008                            * 
*                                                                     *
*          http://sws-challenge.org/wiki/index.php/EON-SWSC2008       *
*                                                                     *
***********************************************************************

GENERAL OVERVIEW

In the successful series of EON and SWS Challenge workshops we intend to
leverage these two initiatives together and to invite researchers and
practitioners from the research areas of Ontologies, Semantic Web and
Semantic Web Services for the Sixth International Workshop on Evaluation
of Ontology-based tools and the Semantic Web Service Challenge
(EON-SWSC2008). Similarly to EON previous workshops, the Semantic Web
Services Challenge aim at evaluating semantic technologies, in this case
those used to mediate, discover and compose Web Services and alike to
EON, the SWSC provides an infrastructure for the development of reusable
technologies. We would like to make a step forward in the evaluation of
the Semantic Web technology by making Semantic Web practitioners face
real problems aimed at real use cases, and the best way to achieve this
is by proposing a technology challenge such as is posed by the SWSC.

Former EON workshops aimed at evaluating ontology-based tools (2002,
2003, and 2004), at evaluating ontologies themselves (2006), and at
evaluating both (2007). Whereas this has provided a meeting point and a
forum for researchers, as ontologies and Semantic Web technologies have
gone beyond the research limits it is required that these evaluations
are available to a broader range of users and developers, both in the
research and in the industrial worlds. Thus the main goal of this
workshop is to lay the foundations for sharing and reusing methods and
tools for Semantic Web technology evaluation (ontology development
tools, ontology merging and alignment tools, ontology-based annotators,
Semantic Web Service technology, etc.).

The large visibility of the Semantic Web already attracts industrial
partners and Semantic Web technology companies are now present in the
industrial world. But the evaluation of the Semantic Web technology and
of the applications that use it is difficult and expensive.
Practitioners want to evaluate their tools and to have benchmark suites
available for doing that in order to minimize the evaluation cost. Also,
there are some benchmark suites that are widely being used by the
community. The problems are that users and developers don't know where
to find these benchmark suites, what these benchmark suites are intended
for, or how to correctly use them.

A well-understood notion of Semantic Web technology evaluation will
foster a consistent level of quality and thus acceptance by industry and
the web community. But to achieve this it is necessary to enable reusing
results and lessons learnt from others.

Semantic Web Services have received a significant amount of attention
and research spending since their beginnings roughly six years ago and
the number of frameworks and algorithms targeted to (semi-) automate
central web service tasks like discovery, matchmaking or composition is
continuously growing. However, there is no scientific method of
comparing the actual functionalities claimed and to comparatively
evaluate the strengths and weaknesses of the various approaches in an
objective way. Furthermore, there is no standard test bed to assess the
robustness and applicability of proposed technologies to real world
problem scenarios. Progress in scientific development and in industrial
adoption is thereby critically hindered.

The Semantic Web Service Challenge is the major initiative dedicated to
work on this problem. The Challenge consists of a set of problem
scenarios to be solved by the participants. The scenarios are organized
in sets of increasingly difficult levels, each designed to focus on a
particular problem aspect. Solutions to the problem scenarios are
presented, discussed and peer reviewed at the SWS-Challenge workshops
and the functional coverage of the solution with regard to the problem
levels is certified at the SWS-Challenge website.

The SWS-Challenge is an ongoing and continuous experiment in developing
a methodology for evaluating the functionality of SWS technologies.
Thus, apart from discussing and evaluating solutions to the Challenge
scenarios, the SWS-Challenge workshops provide the forum to advance the
Challenge itself. In this aspect the Challenge seeks proposals of new
scenarios to be incorporated into the official SWS-Challenge test bed
and fosters discussion on the Challenge’s evaluation methodology as
such. Insights gained during the workshops will provide the necessary
input to the W3C SWS Testbed Incubator Group which aims at standardizing
an evaluation methodology for Semantic Web Services technology.

EON-SWSC2008 will be a two-day workshop: The first day will be dedicated
to ontology and semantic Web evaluation in general (in the spirit of
EON) whereas the second day will focus on SWS technology evaluation (in
the spirit of the SWS-Challenge). Based on the different solutions to a
common application, the workshops will provide a forum for participants
to develop a thorough understanding of each other’s technologies that
greatly exceeds the level of insight that can be reached through the
study of academic papers alone.

TOPICS

The main track of the workshop is focused on the evaluation of Semantic
Web tools and applications, including but not limited to:

* Semantic Web technology evaluation methods
* Methods to describe evaluation aspects and parameters
* Identification of Resource consumption characteristics
* Models for describing and measuring in evaluation
* Techniques and formulas for evaluation
* Infrastructures and tools for evaluation
* Certification and interoperability of tools
* Performance and scalability evaluations and benchmarks
* Integration of evaluation tools into frameworks
* Web scale distribution of Semantic Web

For the SWS-Challenge the workshop seeks novel papers:
* describing implemented solutions to the SWS-Challenge problem
scenarios (SWS-Solution Papers, see below),
* describing other benchmarks targeted at the evaluation of SWS technology,
* dealing with SWS technology evaluation in theory, and
* describing application of SWS technology in industry or on real world
use cases.

Papers describing solutions to SWS Challenge problems must demonstrate
completion of at least one problem level in order to qualify for the
workshop. In order to be successfully certified, a solution has to send
the correct sequence of SOAP-messages to the SWS Challenge test bed. The
solution code must be demonstrated at the workshop and will be evaluated
by a group of workshop organizers and participants.

KEY DATES

Full Research Paper submission: March 23, 2008
Short SWSC-Solution Papers:     March 23, 2008
Acceptance Notification:        April 11, 2008
Research Paper Camera Ready:    April 30, 2008
Full SWSC-Solution Papers:      May 18, 2008

Workshop dates:                 June 1-2, 2008

For SWSC-Solution papers we require a short paper (<= 4 pages)
describing the planned solution by March 23th and a full paper
describing the *implemented solution* by May 18th. SWSC-Solution full
papers will be included in the proceedings, if the claims made in the
paper are verified at the workshop.

SUBMISSION

Submissions are to be formatted according to the guidelines of the
ESWC2008 conference. The page limit for full papers is 10 pages, for
SWS-Challenge solution papers two additional pages are accepted.
Submission details will be published at the workshop website.

All contributions will be peer reviewed by a program committee that will
incorporate well recognized experts.

WORKSHOP ORGANISING COMMITTEE

* Raúl García-Castro, Ontology Engineering Group at Universidad
Politécnica de Madrid (ES)
* Asunción Gómez-Pérez, Ontology Engineering Group, at Universidad
Politécnica de Madrid (ES)
* Charles Petrie, Logic Group at Stanford University (USA)
* Emanuele Della Valle, CEFRIEL (IT)
* Ulrich Küster, Friedrich Schiller University Jena (DE)
* Omair Shafiq, Semantic Technology Institute Innsbruck (AT)
* Michal Zaremba, Semantic Technology Institute Innsbruck (AT)

PROGRAM COMMITTEE

* John Domingue, Knowledge Media Institute, The Open University (UK)
* Tiziana Margaria, Computer Science Dept., U. Potsdam (DE)
* Axel Polleres, DERI, NUIG (IE)
* Richard Waldinger, SRI (USA)
* Peter Emmel, SAP (DE)
* Tomas Vitvar, DERI, NUIG (IE)
* Christoph Bussler, Merced Systems, Inc. (USA)
* Jos de Bruij, KRDB, U. Bozen-Bolzano (IT)
* Alexander Wahler STI Internat. (AT)
* Thorsten Liebig, Ulm University (DE)
* Baoshi Yan, Bosch (USA)
* Oscar Corcho, Universidad Politecnica de Madrid (ES)
* David Martin, SRI (USA)
* more to be announced

FURTHER INFORMATION

For further information refer to the workshop website at

http://sws-challenge.org/wiki/index.php/EON-SWSC2008

-- 

Raúl García Castro
http://delicias.dia.fi.upm.es/~rgarcia/

Ontology Engineering Group (http://www.oeg-upm.net/)
Universidad Politécnica de Madrid
Campus de Montegancedo, s/n - Boadilla del Monte - 28660 Madrid
Phone: +34 91 336 36 70 - Fax: +34 91 352 48 19

Received on Tuesday, 29 January 2008 08:38:28 UTC