Re: Socio technical/Qualitative metrics for LD Benchmarks

HI Denny

Thank you for reply-

I am not sure whether calling the drivers of the Semantic Web
> narrow-minded is exactly helpful in furthering your cause.
>

I am aware such phrasing may not be correct, its difficult to find the
words , but the statement refers to
the narrow vision of their field that some CS have
(if you read again, you will see that unless I am mistaken I have not said
narrow-minded)

These views are not just mine, are widespread and share by many semantic
web researchers who have become aware  that there is a problem within the
research funding schemes. This is why we have large consortia, to try to
bring a diversity of views

However a few influental people with a  narrow vision of the world (and of
the semantic web) seem to be inhibiting the wider views, held by the
majority, to come through and shape research projects and consequent
outcomes

>
> Also, besides stating the obvious, i.e. that we need metrics which are
> not purely technical (something that has been recognized in the field
> since the beginning),


it is not obvious for everyone ,  one particular member of the consortium
actually thinks these issues are not within scope (believe it or not)
and has been sending intimidating requests trying to prevent me from taking
LDBenchmarking discussions in that direction



> it would have been nice if you would have made
> suggestions for the not-so-obvious as well, like, which metrics could be
> used in addition to the currently employed.


 I am working on this Denny, this is precisely the kind of work that I am
trying to get the consortium to agree to do, since there is public funding
allocated to do this. The project was only announced a few days ago, how do
you expect me to do have done the work already?

The consortium partners I have spoken with,  agree with this,  and they too
think it is 'obvious'. But the fact that the person in charge of funding
says this is not within the scope  is challenging-



> That is the hard part.
> I would expect that a well written suggestions or even evaluation of
> such metrics would have quite some impact on the whole field.
>


Very glad you think that Denny. As you know, writing up a   ground breaking
research takes months. While I have expertise and work done in the area, I
would like to write up such a paper as part of the LDBC project and
hopefully you can peer review it
if I manage to drill some sense into these people


Best

PDM

>
>
>
> 2012/11/20 Paola Di Maio <paola.dimaio@gmail.com>:
> >
> > Yesterday I attended the LDBC Technical User Group in Barcelona. This is
> a
> > quick field note for those who are also interested in linked data
> > benchmarking and the progress being made in the field.
> >
> > It was good to meet some of the people I have exchanged  only via email
> so
> > far, and so many socio-technical dimensions crop up in the many
> > presentations. It would important to develop a Benchmark (or set of
> > benchmarks) capable of capturing and measuring them. I suggested that:
> >
> > - technical performance is an emergent property of a socio-technical
> system
> > - vast quantity of triples are a waste of cyber space if they cannot
> produce
> > measurable knowledge advantage (ther is a cognitive dimension to linked
> data
> > outputs)
> > -  I'd likpropose the inclusion of socio-technical/qualitative metrics to
> > the Benchmark, in addition to purely technical /quantitative ones, to
> ensure
> > the usefulness of the latter.
> >
> > I had several conversations with consortium members, and they all seem to
> > agree with this requirement,
> >
> > However the very few members of the consortium with a narrow computer
> > science background may not immediately grasp the socio-technical aspects
> of
> > technical complexity, since the semantic web research and development
> have
> > been, until very recently,  driven by narrow minded computer scientists
> and
> > characterized by the lack of  socio technical vision.
> >
> >
> >
> > Paola Di Maio
> >
> >
> >
> >
>
>
>
> --
> Project director Wikidata
> Wikimedia Deutschland e.V. | Obentrautstr. 72 | 10963 Berlin
> Tel. +49-30-219 158 26-0 | http://wikimedia.de
>
> Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e.V.
> Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg
> unter der Nummer 23855 B. Als gemeinnützig anerkannt durch das
> Finanzamt für Körperschaften I Berlin, Steuernummer 27/681/51985.
>