- From: Paola Di Maio <paola.dimaio@gmail.com>
- Date: Thu, 3 Sep 2009 04:47:02 -0700
- To: Raúl García Castro <rgcmme@gmail.com>
- Cc: "[ontolog-forum]" <ontolog-forum@ontolog.cim3.net>, semantic-web at W3C <semantic-web@w3c.org>
- Message-ID: <4a4804720909030447m1a3dcd2cjd291ae197b137938@mail.gmail.com>
Raul just following up with some thoughts (and to make my point inequivocable) its critical to the validity of an evaluation to capture all the evaluation criteria objectively. No single individual or consortium can be trusted to produce an impartial evaluation mechanism especially if the same researcher/consortium are heavily invovled in the developmet of semantic technologies. So the SEALS effort looks great, and it must an 'open' one to maximise its validity and usefulness to the community at large (other than becoming an instrument to self validate the stuff that one has produced) I look forward to be making a contribution where possible cheers PDM On 9/3/09, Raúl García Castro <rgcmme@gmail.com> wrote: > > Dear Paola, > > If you are interested in the evaluation of semantic technologies, take a > look at the future outcomes of the SEALS European project ( > http://www.seals-project.eu/). > > In this project we aim to build an extensible infrastructure for the > evaluation of different types of semantic technologies: > .- ontology engineering tools > .- storage and reasoning systems > .- matching tools > .- semantic search tools > .- semantic web service tools > > Kind regards, > > On Sun, Aug 9, 2009 at 7:03 PM, Paola Di Maio <paola.dimaio@gmail.com>wrote: > >> Greetings good people! >> >> I am attempting to systematically aggregate as many independent >> evaluations of semantic software tools as possible >> >> (Note: by independent it is intended not carried out by people who are >> paid to do so, or have other vested interests - such as the >> devleopers or consultants themselves or their friends and familieis - >> but by unaffiliated users who are more suitable to appreciate its >> benefits and shorcomings) >> >> We are particularly interested initially in the software developed >> with EU funding (FP6 and FP7 ) >> >> The main frameworks of reference are standard software project >> evaluation methods, including >> >> >> EVALUATION AND RANKING OF ONTOLOGY CONSTRUCTION TOOLS (various >> references included there) >> Md. Ahsan-ul Murshed and Ramanjit Singh >> http://eprints.biblio.unitn.it/archive/00000747/01/013.pdf >> >> >> I have already contacted offlist some of the individuals who have >> offered knowledge and views, and I am now seeking additional wider >> general input on >> >> 1. developing and customizing the review criteria and methodology >> (so that we can format the distributed evaluations using a common >> template) >> >> 2. any volunteered inputs in terms of reviews of software and other >> deliverables (priority is to assess cost/benefit ratio, measured in >> terms of >> functionality and usefulness) >> >> The final goal of this exercise is to contribute to improve the >> effectiveness of EU funding process in semantic web, as well as to >> general >> software output at large, as well as to come up with recommendations. >> >> >> Your contribution can be submitted authored or anonymous (subject to >> verification) >> >> >> Please contact me offlist should you wish to collaborate on this project, >> >> cheers >> >> Paola Di Maio >> > > -- > > Dr. Raúl García Castro > http://delicias.dia.fi.upm.es/~rgarcia/<http://delicias.dia.fi.upm.es/%7Ergarcia/> > > Ontology Engineering Group (http://www.oeg-upm.net/) > Universidad Politécnica de Madrid > Campus de Montegancedo, s/n - Boadilla del Monte - 28660 Madrid > Phone: +34 91 336 36 70 - Fax: +34 91 352 48 19 > -- Paola Di Maio ************************************************** Networked Research Lab, UK ***************************************************
Received on Thursday, 3 September 2009 11:47:46 UTC