Re: R: sws matchmaker contest

dear all,

thanks for the very useful feedback and hints to
ongoing matchmaker development work so far!!

one particular consequence of some of terry's notes,
with which i agree, in essence, would be to build up
a large sws retrieval test collection including domain(-independent)
sws and user queries with subjectively defined relevance sets.
"Subjectively" implies to predominantly involve the potential
business domain service users in the iterative development process.
the hope is that such largely user driven design of a test
collection, in return, strongly influences the development of
"pragmatically usable" sws matchmakers with "reasonably" good 
recall/precision performance on such a collection.

i admit that this process does not, however, also automatically
lead to any solution of the problems related to "How can the user
understand *why* the matchmaker returns what services?"
as Terry also noted in his last email.

but it might be worth to start with the "(user) requirement analysis
phase for sws matchmakers, brokers, search engines". this is what I
would expect to be a joint complementary action at the matchmaker
contest meeting, basically triggering (or prepare to trigger) the same
kind of iterative user-research feedback driven development process as
happened with TREC a few decades ago.

cordial regards, matthias

__________________________________________________
Dr. Matthias Klusch
German Research Center for Artificial Intelligence
Stuhlsatzenhausweg 3
66123 Saarbruecken, Germany
Phone: +49-681-302-5297, Fax: +49-681-302-2235
http://www.dfki.de/~klusch/, klusch@dfki.de
__________________________________________________

Received on Friday, 25 August 2006 12:34:51 UTC