- From: Raúl García Castro <rgarcia@delicias.dia.fi.upm.es>
- Date: Mon, 24 Mar 2008 10:58:59 +0100
- To: semantic-web@w3.org
(Apologies for multiple postings) *Deadline extended - 27th March* *********************************************************************** * * * Call for papers * * * * Sixth International Workshop on Evaluation of * * Ontology-based tools and the Semantic Web * * Service Challenge (EON-SWSC2008) * * at the 5th European Semantic Web Conference (ESWC2008) * * Sheraton La Caleta, Tenerife, Spain * * June 2nd-3rd, 2008 * * * * http://sws-challenge.org/wiki/index.php/EON-SWSC2008 * * * *********************************************************************** GENERAL OVERVIEW In the successful series of EON and SWS Challenge workshops we intend to leverage these two initiatives together and to invite researchers and practitioners from the research areas of Ontologies, Semantic Web and Semantic Web Services for the Sixth International Workshop on Evaluation of Ontology-based tools and the Semantic Web Service Challenge (EON-SWSC2008). Similarly to EON previous workshops, the Semantic Web Services Challenge aim at evaluating semantic technologies, in this case those used to mediate, discover and compose Web Services and alike to EON, the SWSC provides an infrastructure for the development of reusable technologies. We would like to make a step forward in the evaluation of the Semantic Web technology by making Semantic Web practitioners face real problems aimed at real use cases, and the best way to achieve this is by proposing a technology challenge such as is posed by the SWSC. Former EON workshops aimed at evaluating ontology-based tools (2002, 2003, and 2004), at evaluating ontologies themselves (2006), and at evaluating both (2007). Whereas this has provided a meeting point and a forum for researchers, as ontologies and Semantic Web technologies have gone beyond the research limits it is required that these evaluations are available to a broader range of users and developers, both in the research and in the industrial worlds. Thus the main goal of this workshop is to lay the foundations for sharing and reusing methods and tools for Semantic Web technology evaluation (ontology development tools, ontology merging and alignment tools, ontology-based annotators, Semantic Web Service technology, etc.). The large visibility of the Semantic Web already attracts industrial partners and Semantic Web technology companies are now present in the industrial world. But the evaluation of the Semantic Web technology and of the applications that use it is difficult and expensive. Practitioners want to evaluate their tools and to have benchmark suites available for doing that in order to minimize the evaluation cost. Also, there are some benchmark suites that are widely being used by the community. The problems are that users and developers don't know where to find these benchmark suites, what these benchmark suites are intended for, or how to correctly use them. A well-understood notion of Semantic Web technology evaluation will foster a consistent level of quality and thus acceptance by industry and the web community. But to achieve this it is necessary to enable reusing results and lessons learnt from others. Semantic Web Services have received a significant amount of attention and research spending since their beginnings roughly six years ago and the number of frameworks and algorithms targeted to (semi-) automate central web service tasks like discovery, matchmaking or composition is continuously growing. However, there is no scientific method of comparing the actual functionalities claimed and to comparatively evaluate the strengths and weaknesses of the various approaches in an objective way. Furthermore, there is no standard test bed to assess the robustness and applicability of proposed technologies to real world problem scenarios. Progress in scientific development and in industrial adoption is thereby critically hindered. The Semantic Web Service Challenge is the major initiative dedicated to work on this problem. The Challenge consists of a set of problem scenarios to be solved by the participants. The scenarios are organized in sets of increasingly difficult levels, each designed to focus on a particular problem aspect. Solutions to the problem scenarios are presented, discussed and peer reviewed at the SWS-Challenge workshops and the functional coverage of the solution with regard to the problem levels is certified at the SWS-Challenge website. The SWS-Challenge is an ongoing and continuous experiment in developing a methodology for evaluating the functionality of SWS technologies. Thus, apart from discussing and evaluating solutions to the Challenge scenarios, the SWS-Challenge workshops provide the forum to advance the Challenge itself. In this aspect the Challenge seeks proposals of new scenarios to be incorporated into the official SWS-Challenge test bed and fosters discussion on the Challenge’s evaluation methodology as such. Insights gained during the workshops will provide the necessary input to the W3C SWS Testbed Incubator Group which aims at standardizing an evaluation methodology for Semantic Web Services technology. EON-SWSC2008 will be a two-day workshop: The first day will be dedicated to ontology and semantic Web evaluation in general (in the spirit of EON) whereas the second day will focus on SWS technology evaluation (in the spirit of the SWS-Challenge). Based on the different solutions to a common application, the workshops will provide a forum for participants to develop a thorough understanding of each other’s technologies that greatly exceeds the level of insight that can be reached through the study of academic papers alone. TOPICS The main track of the workshop is focused on the evaluation of Semantic Web tools and applications, including but not limited to: * Semantic Web technology evaluation methods * Methods to describe evaluation aspects and parameters * Identification of Resource consumption characteristics * Models for describing and measuring in evaluation * Techniques and formulas for evaluation * Infrastructures and tools for evaluation * Certification and interoperability of tools * Performance and scalability evaluations and benchmarks * Integration of evaluation tools into frameworks * Web scale distribution of Semantic Web For the SWS-Challenge the workshop seeks novel papers: * describing implemented solutions to the SWS-Challenge problem scenarios (SWS-Solution Papers, see below), * describing other benchmarks targeted at the evaluation of SWS technology, * dealing with SWS technology evaluation in theory, and * describing application of SWS technology in industry or on real world use cases. Papers describing solutions to SWS Challenge problems must demonstrate completion of at least one problem level in order to qualify for the workshop. In order to be successfully certified, a solution has to send the correct sequence of SOAP-messages to the SWS Challenge test bed. The solution code must be demonstrated at the workshop and will be evaluated by a group of workshop organizers and participants. KEY DATES Full Research Paper submission: March 27, 2008 Short SWSC-Solution Papers: March 27, 2008 Acceptance Notification: April 11, 2008 Research Paper Camera Ready: April 30, 2008 Full SWSC-Solution Papers: May 18, 2008 Workshop dates: June 2-3, 2008 For SWSC-Solution papers we require a short paper (<= 4 pages) describing the planned solution by March 23th and a full paper describing the implemented solution by May 18th. SWSC-Solution full papers will be included in the proceedings, if the claims made in the paper are verified at the workshop. SUBMISSION Submissions are to be formatted according to the guidelines of the ESWC2008 conference. The page limit for full papers is 10 pages, for SWS-Challenge solution papers two additional pages are accepted. Papers are to be submitted online via the EONSWSC 2008 EasyChair submission system: http://www.easychair.org/conferences/?conf=eonswsc2008 Submission details are published at the workshop website. All contributions will be peer reviewed by a program committee that will incorporate well recognized experts. WORKSHOP ORGANISING COMMITTEE * Raúl García-Castro, Ontology Engineering Group at Universidad Politécnica de Madrid (ES) * Asunción Gómez-Pérez, Ontology Engineering Group, at Universidad Politécnica de Madrid (ES) * Charles Petrie, Logic Group at Stanford University (USA) * Emanuele Della Valle, CEFRIEL (IT) * Ulrich Küster, Friedrich Schiller University Jena (DE) * Omair Shafiq, Semantic Technology Institute Innsbruck (AT) * Michal Zaremba, Semantic Technology Institute Innsbruck (AT) PROGRAM COMMITTEE * John Domingue, Knowledge Media Institute, The Open University (UK) * Tiziana Margaria, Computer Science Dept., U. Potsdam (DE) * Axel Polleres, DERI, NUIG (IE) * Richard Waldinger, SRI (USA) * Peter Emmel, SAP (DE) * Tomas Vitvar, DERI, NUIG (IE) * Christoph Bussler, Merced Systems, Inc. (USA) * Jos de Bruij, KRDB, U. Bozen-Bolzano (IT) * Alexander Wahler STI Internat. (AT) * Thorsten Liebig, Ulm University (DE) * Baoshi Yan, Bosch (USA) * Oscar Corcho, Universidad Politecnica de Madrid (ES) * David Martin, SRI (USA) FURTHER INFORMATION For further information refer to the workshop website at http://sws-challenge.org/wiki/index.php/EON-SWSC2008 -- Raúl García Castro http://delicias.dia.fi.upm.es/~rgarcia/ Ontology Engineering Group (http://www.oeg-upm.net/) Universidad Politécnica de Madrid Campus de Montegancedo, s/n - Boadilla del Monte - 28660 Madrid Phone: +34 91 336 36 70 - Fax: +34 91 352 48 19
Received on Monday, 24 March 2008 10:01:38 UTC