Re: Socio technical/Qualitative metrics for LD Benchmarks

Hello,
Well there is an issue about narrow mindedness and scope and I am going to
iterate the obvious here.
Technologies thrive on detail and people mastering that detail. With the
mastering of that detail some technologists are able to make surprising and
new connections (what ever surprising and new is, it is relative).
Most cannot, due to so many factors, one large one being the momentum of
this technology, that is the current of the river in which they swim.
People who lead working groups, especially newly formed ones, want people
to hitch up with them while generally keeping in the (technical) stream.
This has, in particular, been the history of internet related technologies.
People want to seem to be forging ahead plus they do not want to be left
behind. I'm sure there is some paradox here :-) .

So, that said, there is much more to be said about the difficulty some find
in thinking about these aspects of qualitative outcome.

I think that the way to approach this is through thinking about quality
with respect to values (made explicit in the research). Those values might
be human, individual cognitive or other psychological, political or other
group as defined by the researcher.

At the moment, as far as I can see, and my view is certainly circumscribed,
those values have been defined by default as those of commerce.
I don't think that postion is quite so acceptable or fashionable now, what
with the financial crash. It is the historical role of academia to rise
above such things.

Good luck Paola!

Adam



On 20 November 2012 16:52, Paola Di Maio <paola.dimaio@gmail.com> wrote:

> Denny
> CORRECTION!
> Just to clarify-
> I just re-read my post and yes I did say narrow-minded in a further
> sentenc, just to clarify, with it intended as 'with narrow vision of the
> research field'.
> P
>
>
> On Tue, Nov 20, 2012 at 2:12 PM, Paola Di Maio <paola.dimaio@gmail.com>wrote:
>
>>
>> HI Denny
>>
>> Thank you for reply-
>>
>> I am not sure whether calling the drivers of the Semantic Web
>>> narrow-minded is exactly helpful in furthering your cause.
>>>
>>
>> I am aware such phrasing may not be correct, its difficult to find the
>> words , but the statement refers to
>> the narrow vision of their field that some CS have
>> (if you read again, you will see that unless I am mistaken I have not
>> said narrow-minded)
>>
>> These views are not just mine, are widespread and share by many semantic
>> web researchers who have become aware  that there is a problem within the
>> research funding schemes. This is why we have large consortia, to try to
>> bring a diversity of views
>>
>> However a few influental people with a  narrow vision of the world (and
>> of the semantic web) seem to be inhibiting the wider views, held by the
>> majority, to come through and shape research projects and consequent
>> outcomes
>>
>>>
>>> Also, besides stating the obvious, i.e. that we need metrics which are
>>> not purely technical (something that has been recognized in the field
>>> since the beginning),
>>
>>
>> it is not obvious for everyone ,  one particular member of the consortium
>> actually thinks these issues are not within scope (believe it or not)
>> and has been sending intimidating requests trying to prevent me from
>> taking LDBenchmarking discussions in that direction
>>
>>
>>
>>> it would have been nice if you would have made
>>> suggestions for the not-so-obvious as well, like, which metrics could be
>>> used in addition to the currently employed.
>>
>>
>>  I am working on this Denny, this is precisely the kind of work that I am
>> trying to get the consortium to agree to do, since there is public funding
>> allocated to do this. The project was only announced a few days ago, how do
>> you expect me to do have done the work already?
>>
>> The consortium partners I have spoken with,  agree with this,  and they
>> too think it is 'obvious'. But the fact that the person in charge of
>> funding says this is not within the scope  is challenging-
>>
>>
>>
>>> That is the hard part.
>>> I would expect that a well written suggestions or even evaluation of
>>> such metrics would have quite some impact on the whole field.
>>>
>>
>>
>> Very glad you think that Denny. As you know, writing up a   ground
>> breaking research takes months. While I have expertise and work done in the
>> area, I would like to write up such a paper as part of the LDBC project and
>> hopefully you can peer review it
>> if I manage to drill some sense into these people
>>
>>
>> Best
>>
>> PDM
>>
>>>
>>>
>>>
>>> 2012/11/20 Paola Di Maio <paola.dimaio@gmail.com>:
>>> >
>>> > Yesterday I attended the LDBC Technical User Group in Barcelona. This
>>> is a
>>> > quick field note for those who are also interested in linked data
>>> > benchmarking and the progress being made in the field.
>>> >
>>> > It was good to meet some of the people I have exchanged  only via
>>> email so
>>> > far, and so many socio-technical dimensions crop up in the many
>>> > presentations. It would important to develop a Benchmark (or set of
>>> > benchmarks) capable of capturing and measuring them. I suggested that:
>>> >
>>> > - technical performance is an emergent property of a socio-technical
>>> system
>>> > - vast quantity of triples are a waste of cyber space if they cannot
>>> produce
>>> > measurable knowledge advantage (ther is a cognitive dimension to
>>> linked data
>>> > outputs)
>>> > -  I'd likpropose the inclusion of socio-technical/qualitative metrics
>>> to
>>> > the Benchmark, in addition to purely technical /quantitative ones, to
>>> ensure
>>> > the usefulness of the latter.
>>> >
>>> > I had several conversations with consortium members, and they all seem
>>> to
>>> > agree with this requirement,
>>> >
>>> > However the very few members of the consortium with a narrow computer
>>> > science background may not immediately grasp the socio-technical
>>> aspects of
>>> > technical complexity, since the semantic web research and development
>>> have
>>> > been, until very recently,  driven by narrow minded computer
>>> scientists and
>>> > characterized by the lack of  socio technical vision.
>>> >
>>> >
>>> >
>>> > Paola Di Maio
>>> >
>>> >
>>> >
>>> >
>>>
>>>
>>>
>>> --
>>> Project director Wikidata
>>> Wikimedia Deutschland e.V. | Obentrautstr. 72 | 10963 Berlin
>>> Tel. +49-30-219 158 26-0 | http://wikimedia.de
>>>
>>> Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e.V.
>>> Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg
>>> unter der Nummer 23855 B. Als gemeinnützig anerkannt durch das
>>> Finanzamt für Körperschaften I Berlin, Steuernummer 27/681/51985.
>>>
>>
>>
>