Re: Where are the Linked Data Driven Smart Agents (Bots) ?

Ruben,

One thing is science. Another is engineering.

Part of the scientific process is defining an experiment and doing the
evaluation. If we don't know the right evaluation metrics (I agree with you
that we don't), then that is the current challenge we, as a semantic web
scientific community, have to tackle. It shouldn't discourage you... on the
contrary, it should encourage you to identify novel ways to evaluate what
you are doing and convince the community why it is important.

In my opinion, evaluation of systems on the web is different that what the
CS community has been used to (CS is a young science). Evaluation has been
on run time, space consumed, precision, recall, sound, completeness. On the
web, those may not be the aspects we want to measures. In the words of Jim
Hendler:

“You want a good thesis? IR is based on precision and recall and the minute
you add semantics, it is a meaningless feature. Logic is based on soundness
and completeness. We don’t want soundness and completeness. We want a few
good answers quickly.”

– Prof. James A. Hendler, 2009, on the topic of answering queries over the
Semantic Web.
https://www.youtube.com/watch?v=sbmMxzOeZ-4

Here is a short position paper that Olaf Hartig and I wrote 5 years ago
(time flies when you are having fun): Towards a Query Language for the Web
of Data (A Vision Paper). http://arxiv.org/abs/1110.3017

My 2cts



--
Juan Sequeda, Ph.D
+1-575-SEQ-UEDA
www.juansequeda.com

On Wed, Jul 6, 2016 at 11:38 AM, Ruben Verborgh <ruben.verborgh@ugent.be>
wrote:

> Hi,
>
> This is a very important question for our community,
> given that smart agents once were an important theme.
> Actually, the main difference we could bring with the SemWeb
> is that our clients could be decentralized
> and actually run on the client side, in contrast to others.
>
> One of the main problems I see is how our community
> (now particularly thinking about the scientific subgroup)
> receives submissions of novel work.
> We have evolved into an extremely quantitative-oriented view,
> where anything that can be measured with numbers
> is largely favored over anything that cannot.
>
> Given that the smart agents / bots field is quite new,
> we don't know the right evaluation metrics yet.
> As such, it is hard to publish a paper on this
> at any of the main venues (ISWC / ESWC / …).
> This discourages working on such themes.
>
> Hence, I see much talent and time going to
> incremental research, which is easy to evaluate well,
> but not necessarily as ground-breaking.
> More than a decade of SemWeb research
> has mostly brought us intelligent servers,
> but not yet the intelligent clients we wanted.
>
> So perhaps we should phrase the question more broadly:
> how can we as a community be more open
> to novel and disruptive technologies?
>
> Best,
>
> Ruben
>