Re: Socio technical/Qualitative metrics for LD Benchmarks

Adam
thanks-
Please note I do not 'complain about the narrow view of the world per se
I am a great advocate of freedom of worldviews :-)
Also, as you also say. some technical advances are by definition 'narrow'
in the sense that one milisecond performance improvement may take a great
deal of work
to achieve, and is by deintion a very small thing, yet a  big deal to some.
I am aware that science can advance in small steps
B ut things are changing
we need to take into account 'acceleration' and a lot of other factors
What I complain about is forcing the scope of a great big project to a
single
narrow view of what is within scope, despite consortium members think
otherwise

I shall try to make other important points I was trying to put across in my
reply to Gio


> I think that the way to approach this is through thinking about quality
> with respect to values (made explicit in the research). Those values might
> be human, individual cognitive or other psychological, political or other
> group as defined by the researcher.
>

that, and probably more

>
> At the moment, as far as I can see, and my view is certainly
> circumscribed, those values have been defined by default as those of
> commerce.
>
not even that. commerce is also 'big' when you count everyone in. I would
say defined by those who dominate the market, and those who have the power
and authortity to decide
a) who gets in and who stays out of a research consortium
b)  who makes decisions and how (ie, a single fascist in high position of
power will make decisions and other consortium members shut up because they
do not want to get in trouble)

I don't think that postion is quite so acceptable or fashionable now, what
> with the financial crash. It is the historical role of academia to rise
> above such things.
>

things have changed. academia is  mostly all about  business nowadays

>
> Good luck Paola!
>

thank you Ada, I need it
see more in reply to Gio


PDM





>
> On 20 November 2012 16:52, Paola Di Maio <paola.dimaio@gmail.com> wrote:
>
>> Denny
>> CORRECTION!
>> Just to clarify-
>> I just re-read my post and yes I did say narrow-minded in a further
>> sentenc, just to clarify, with it intended as 'with narrow vision of the
>> research field'.
>>  P
>>
>>
>> On Tue, Nov 20, 2012 at 2:12 PM, Paola Di Maio <paola.dimaio@gmail.com>wrote:
>>
>>>
>>> HI Denny
>>>
>>> Thank you for reply-
>>>
>>> I am not sure whether calling the drivers of the Semantic Web
>>>> narrow-minded is exactly helpful in furthering your cause.
>>>>
>>>
>>> I am aware such phrasing may not be correct, its difficult to find the
>>> words , but the statement refers to
>>> the narrow vision of their field that some CS have
>>> (if you read again, you will see that unless I am mistaken I have not
>>> said narrow-minded)
>>>
>>> These views are not just mine, are widespread and share by many semantic
>>> web researchers who have become aware  that there is a problem within the
>>> research funding schemes. This is why we have large consortia, to try to
>>> bring a diversity of views
>>>
>>> However a few influental people with a  narrow vision of the world (and
>>> of the semantic web) seem to be inhibiting the wider views, held by the
>>> majority, to come through and shape research projects and consequent
>>> outcomes
>>>
>>>>
>>>> Also, besides stating the obvious, i.e. that we need metrics which are
>>>> not purely technical (something that has been recognized in the field
>>>> since the beginning),
>>>
>>>
>>> it is not obvious for everyone ,  one particular member of the
>>> consortium actually thinks these issues are not within scope (believe it or
>>> not)
>>> and has been sending intimidating requests trying to prevent me from
>>> taking LDBenchmarking discussions in that direction
>>>
>>>
>>>
>>>> it would have been nice if you would have made
>>>> suggestions for the not-so-obvious as well, like, which metrics
>>>> could be used in addition to the currently employed.
>>>
>>>
>>>  I am working on this Denny, this is precisely the kind of work that I
>>> am trying to get the consortium to agree to do, since there is public
>>> funding allocated to do this. The project was only announced a few days
>>> ago, how do you expect me to do have done the work already?
>>>
>>> The consortium partners I have spoken with,  agree with this,  and they
>>> too think it is 'obvious'. But the fact that the person in charge of
>>> funding says this is not within the scope  is challenging-
>>>
>>>
>>>
>>>> That is the hard part.
>>>> I would expect that a well written suggestions or even evaluation of
>>>> such metrics would have quite some impact on the whole field.
>>>>
>>>
>>>
>>> Very glad you think that Denny. As you know, writing up a   ground
>>> breaking research takes months. While I have expertise and work done in the
>>> area, I would like to write up such a paper as part of the LDBC project and
>>> hopefully you can peer review it
>>> if I manage to drill some sense into these people
>>>
>>>
>>> Best
>>>
>>> PDM
>>>
>>>>
>>>>
>>>>
>>>> 2012/11/20 Paola Di Maio <paola.dimaio@gmail.com>:
>>>> >
>>>> > Yesterday I attended the LDBC Technical User Group in Barcelona. This
>>>> is a
>>>> > quick field note for those who are also interested in linked data
>>>> > benchmarking and the progress being made in the field.
>>>> >
>>>> > It was good to meet some of the people I have exchanged  only via
>>>> email so
>>>> > far, and so many socio-technical dimensions crop up in the many
>>>> > presentations. It would important to develop a Benchmark (or set of
>>>> > benchmarks) capable of capturing and measuring them. I suggested that:
>>>> >
>>>> > - technical performance is an emergent property of a socio-technical
>>>> system
>>>> > - vast quantity of triples are a waste of cyber space if they cannot
>>>> produce
>>>> > measurable knowledge advantage (ther is a cognitive dimension to
>>>> linked data
>>>> > outputs)
>>>> > -  I'd likpropose the inclusion of socio-technical/qualitative
>>>> metrics to
>>>> > the Benchmark, in addition to purely technical /quantitative ones, to
>>>> ensure
>>>> > the usefulness of the latter.
>>>> >
>>>> > I had several conversations with consortium members, and they all
>>>> seem to
>>>> > agree with this requirement,
>>>> >
>>>> > However the very few members of the consortium with a narrow computer
>>>> > science background may not immediately grasp the socio-technical
>>>> aspects of
>>>> > technical complexity, since the semantic web research and development
>>>> have
>>>> > been, until very recently,  driven by narrow minded computer
>>>> scientists and
>>>> > characterized by the lack of  socio technical vision.
>>>> >
>>>> >
>>>> >
>>>> > Paola Di Maio
>>>> >
>>>> >
>>>> >
>>>> >
>>>>
>>>>
>>>>
>>>> --
>>>> Project director Wikidata
>>>> Wikimedia Deutschland e.V. | Obentrautstr. 72 | 10963 Berlin
>>>> Tel. +49-30-219 158 26-0 | http://wikimedia.de
>>>>
>>>> Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e.V.
>>>> Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg
>>>> unter der Nummer 23855 B. Als gemeinnützig anerkannt durch das
>>>> Finanzamt für Körperschaften I Berlin, Steuernummer 27/681/51985.
>>>>
>>>
>>>
>>
>