Re: independent semantic software evaluation frameworks?

Thanks to all for replies!
see notes below

Adrian:
There are a few that I am personally trying to run, with varying degrees of
success, but any tool that people want to review from the pool below, for
example. Will circulate evaluation sheet to those who want do run something
and contact me offlist pls.  At this stage we prioritize publicly funded
ones, rather than commercial tools.

http://cordis.europa.eu/info-management/.

http://cordis.europa.eu/ist/kct/fp6_projects_semantic.htm.

http://cordis.europa.eu/fp7/ict/content-knowledge/projects_en.html.

Richard:
I am not adding editors to any list, and again, prioritizing publicly
funded. i dont think your tool is?

Pieter:
Yes, thanks! will follow up and ....

"In any case the problem is rather social/cultural/organisational than
merely technical."

so true... for some of us, technology is mainly a mean to solve real world
problems in real world context, so any technical issue is, at least in part,
determined by social/cultural/organisational issues. This is often
forgotten, and technology is done for technology's sake.

Chris:
Thanks a lot for spreading the word
Even more value would come from this being a public , open, collaborative
effort......

I am still working on the evaluation sheet, would welcome thoughts and
criticism, so please get in touch  if you are interested

cheers

PDM




On Sun, Aug 9, 2009 at 6:03 PM, Paola Di Maio<paola.dimaio@gmail.com> wrote:
> Greetings good people!
>
> I am attempting to systematically aggregate as many independent
> evaluations of semantic software tools as possible
>
> (Note: by independent it is intended not carried out by people who are
> paid to do so, or have other vested interests  - such as the
> devleopers or consultants themselves  or their friends and familieis -
>  but by unaffiliated users who are more suitable to appreciate its
> benefits and shorcomings)
>
> We are particularly interested initially in the software developed
> with EU funding (FP6 and FP7 )
>
> The main frameworks of reference are standard software project
> evaluation methods, including
>
>
> EVALUATION AND RANKING OF ONTOLOGY CONSTRUCTION TOOLS (various
> references included there)
> Md. Ahsan-ul Murshed and Ramanjit Singh
> http://eprints.biblio.unitn.it/archive/00000747/01/013.pdf
>
>
> I have already contacted offlist some of the individuals who have
> offered knowledge and views, and I am now  seeking additional wider
> general  input on
>
> 1.  developing and customizing the  review criteria  and methodology
> (so that we can format the distributed evaluations using a common
template)
>
> 2.  any volunteered inputs in terms of reviews of software and other
> deliverables (priority is to assess  cost/benefit ratio, measured in
> terms of
> functionality and usefulness)
>
> The final goal of this exercise is to contribute  to improve the
> effectiveness of EU funding process in semantic web, as well as to
> general
> software output at large, as well as to come up with  recommendations.
>
>
> Your contribution can be submitted authored or anonymous (subject to
> verification)
>
>
> Please contact me offlist should you wish to collaborate on this project,
>
> cheers
>
> Paola Di Maio
>
>
>
>
>
> --
> Paola Di Maio
> **************************************************
> Strategic Advisor
> Networked Research Lab, UK
> **************************************************
>



-- 
Paola Di Maio
**************************************************
DMEM (Design, Manufacture, Engineering, Management)
University of  Strathclyde
Room 106, 75 Montrose Street
Glasgow G1 1XJ UK