CfP: Empirical 2014 Workshop on statistical methodology in Semantic Web research

                First International Workshop-Empirical 2014   
                       May 25, 2014, Crete, Greece.
               At the 11th Extended Semantic Web Conference 
                               (ESWC 2014)
                       http://2014.empirical-ws.org/

Experimental work in the Semantic Web research typically aims to evaluate 
proposed scientific contributions on existing benchmarks. Contributions are 
newly designed artifacts such as systems or algorithms, which need to be 
evaluated in terms of their overall utility or performance. In this, 
experimentation in our discipline can be seen as part of the build and 
evaluate cycle that also characterizes the design research in other 
engineering sciences. In most disciplines in the natural sciences, however, 
experimentation typically aims to justify the proposed scientific contribution. 
This contribution is a theoretical explanation, which gives rise to hypotheses 
of which the truth needs to be tested in the real world. Typically, such tests 
have to deal with various forms of variability, forcing researchers to apply 
statistical inference techniques to draw conclusions about their hypotheses 
from their research data. New contribution typically require new samples of 
data to be collected and evaluated.

This workshop starts from the assumption that the Semantic Web research would 
progress faster and could broaden its scope when our community embraces both 
evaluation and explanation and make them reinforce one another. However, we 
observe that the build and evaluate cycle is currently overly represented in 
our field, while there is insufficient attention for the research cycle of 
theorize and justify. Thus, reproducibility and generality of empirical 
results presented in the literature cannot be commonly ensured. The workshop 
aims bringing together researchers interested in the formalization of 
statistical methods to design and analyze empirical studies of Semantic Web 
technologies. We welcome any contribution that relates directly to this 
observation, and that allows to illustrate the effects of the lack justification 
and explanation in the generality of reported experimental results.  We invite 
the submissions of research papers (max. 15 pages) or vision and position 
papers (max. 8 pages).

Potential topics include, but are not limited to:
-------------------------------------------------

-) Foundations of hypothesis formulation: triangulation, common constructs in 
the 
Semantic Web (e.g., performance, completeness) and their validity, typical 
operationalizations, and quantifiability.
-) Statistical methods to demonstrate significance and levels of confidence of 
experimental studies on Semantic Web technologies and empirical results.
-) Experimental methodologies to design reproducible experimental studies in 
the context of Semantic Web technologies.
-) Critical examinations of assumptions inherent in statistical practices in 
Semantic Web research (e.g., assuming normal distribution of populations).
-) Limitations of existing statistical methods on empirical studies of Big 
Data.
-) Statistical techniques to analyze and improve benchmarks.

Workshop key dates
-----------------------
Full Paper Submission Deadline March 6, 2014
Acceptance notification: April 1, 2014
Camera ready version: April 15, 2014
Workshop day: May 25, 2014, morning session

Submissions Guidelines
------------------------
Research papers are limited to 15 pages, vision papers to 8 pages, and 
witnesses to the trial are informal submissions.
All papers will be formatted according to the LNCS format.
Submissions will only be accepted through Easychair 
https://www.easychair.org/conferences/?conf=empirical2014

Publication
-------------------------
Proceedings of the workshop will be published online. All papers presented at 
the workshop will be invited to be revised and extended for a second peer-
review process. Accepted papers will be published in a Special Issue of Open 
Journal of Semantic Web (OJWS)
http://www.ronpub.com/index.php/journals/ojsw.


Workshop Chairs
-----------------------
Kjetil Kjernsmo, University of Oslo, Norway.
Maria-Esther Vidal, Universidad Simon Bolivar, Venezuela.
Jacco van Ossenbruggen, VU University Amsterdam, Netherlands.

Program Committee
_________________
Maribel Acosta, AIFB, Karlsruhe Institute of Technology, Germany.
Soren Auer, University of Bonn, Germany.
Oscar Corcho, Universidad Politecnica de Madrid, Spain.
Emanuele Della Valle, Politecnico di Milano, Italy.
Fabian Flöck, AIFB, Karlsruhe Institute of Technology, Germany.
Martin Giese, University of Oslo, Norway.
Peter Haase, fluid Operations AG, Germany.
Tomi Kauppinen, Aalto University School of Science, Finland.
Manolis Koubarakis, University of Athens, Greece.
Gabriela Montoya, University of Nantes, France.
Jose Mora, Universidad Politecnica de Madrid, Spain.
Daniel Palhazi-Cuervo, Universiteit of Antwerpen, Belgium.
Josiane Xavier Parreira, Digital Enterprise Research Institute (DERI), and 
National University of Ireland, Irland.
Axel Polleres, Vienna University of Economics & Business, Austria.
Ansgar Scherp, Kiel University, Germany.
Willem van Hage, VU University Amsterdam, Netherlands.
Frank van Harmelen, VU University Amsterdam, Netherlands.
John Tyssedal, Norwegian University of Science and Technology, Norway.
Matthias Thimm, Universität Koblenz, Germany.

Received on Tuesday, 14 January 2014 22:44:31 UTC