Re: Seeking Help with finding an assertion

The costs of not knowing is of course the reason for investing in science, 
and is for practical purposes infinite. Wouldn't it be nice if it ended 
there, but we also must deal with spin, turf protection, disinformation, and 
digital replication of false information..... In extreme examples of 
digitally enhanced negative information that transforms into negative 
knowledge (belief systems)- terrorism and false health cures are good 
examples, but it's everywhere.

Unfortunately those costs are very challenging to recover even when systems 
are built that can do so. For example, Google provides excellent search 
technology, and is making billions from the advertising model only because 
of the generosity (or other factors) of hundreds of millions of people. They 
do have a revenue sharing program, but it doesn't come close to covering a 
fraction of the cost for generating that information, so we have a problem 
long term.

That's a major issue with free- a very small portion of the population is 
being compensated, it's not always the best or even accurate information, 
and even when coming from the most reputable institutions- it is often 
conflicted- an old media issue as well. So architects need to redesign 
incentives and try to keep our emotion based best intentions checked, if we 
want sustainable systems that are properly aligned. That's one major issue 
we've been working on. Call it economic justice. But with the public 
Internet combined with democracy, these issues may eventually mean the 
difference between survival or not.

Another is that the semantic web should have been the semantic internet 
because the web, while very important, isn't totally integrated with 
messaging, which is a major source of information overload and bad 
information that often leads to corrupt knowledge, replication of 
falsehoods, etc. Even false rumors within organizations and communities is 
spread rapidly now and causing major problems just in the small slices I 
see. Then add the cost of missing just one essential message, or an 
organization not giving it the priority deserved for whatever reason- like 
the infamous Phoenix memo in intelligence that if properly handled could 
have prevented 9/11.

So while an essential piece of the puzzle, I think too much emphasis has 
been placed on ontologies for the past decade, and far too many view the 
sector as a cure all. Very important component in particular for life 
sciences, and accelerated discovery is taking place, but every path I've 
gone down suggests strongly that unless each of the areas are addressed, 
particularly for organizations, systemic failure (at least events if not 
continual) is probable, which is why we take a holistic approach.

One problem I am still seeing with ontological languages and tools 
surrounding them is that they are not sufficiently flexible to deal with the 
rapid transition and growth of knowledge. Many if not most areas of learning 
are evolving very quickly- yesterday's certainty is today's uncertainty and 
tomorrow's epiphany. Here we may have a conflict within the very community 
as this situation creates good job protection as XML did, but also serves to 
limit adoption and usefulness, which is why we are focused on quality in, 
quality out. ERP has a similar structural problem that I for one would like 
to avoid.

Prevention is the best medicine in IT architecture as well as healthcare, 
and also generates the highest ROI, even if so few are aware of it.

.02-

Mark Montgomery
CEO, Kyield
http://www.kyield.com
Managing Partner
Initium Venture Capital
http://www.initiumcapital.com









----- Original Message ----- 
From: "Skinner, Karen (NIH/NIDA) [E]" <kskinner@nida.nih.gov>
To: "Kei Cheung" <kei.cheung@yale.edu>
Cc: "public-semweb-lifesci hcls" <public-semweb-lifesci@w3.org>
Sent: Thursday, July 05, 2007 11:39 PM
Subject: RE: Seeking Help with finding an assertion



Many thanks to all for this lively discussion, the helpful references,
and your generosity with your knowledge!

I could not access the video presentation yesterday but was finally
successful late tonight. It indeed was very interesting.  The paper:
"Understanding user goals in web search" also appears really relevant,
but it is not readily accessible through the NIH online publications.
I look forward to going through the other references, and your comments
about them have been helpful already.

Just a comment about "negative knowledge." I did not know it has any
sort of formal meaning. I coined the phrase for my own purpose in
reference to a situation where some information might exist, but a
potential user might not be aware of it. For example, a consumer could
go to a local store and compare prices for refrigerators. But if the
consumer visited more stores, she could learn even more about prices and
models. If she visited only one store, all other information about
prices would be "negative" knowledge to her, because it does not exist
for her -- i.e., she does not know about it. Certainly, at some point,
the cost of expenditure of effort to "know" exceeds the benefit, whether
that cost is determined by an hourly wage equivalent, or some subjective
measure of the value of her time.

In science, such an analysis quickly becomes very complex. In some
cases, an investigator may not care if a certain study has been
conducted because they only trust the reagents or data they themselves
generate, and the existence of data and resources is irrelevant to that
investigator.

On the other hand, suppose that "database X" did not exist, but the
existence of information that would have been found in it can be
identified and obtained only through locating and reading thousands of
individual papers. At what point does the cost of locating and reading
the papers by "y" number of users exceed the cost of the database? It
would seem that most of the cost would derive from the expense of
determining IF the knowledge existed. How many papers would the
scientist have to read before being certain the knowledge or data did
not exist?


Karen Skinner

-----Original Message-----
From: Kei Cheung [mailto:kei.cheung@yale.edu]
Sent: Thursday, July 05, 2007 10:36 PM
To: Chris Mungall
Cc: Skinner, Karen (NIH/NIDA) [E]; public-semweb-lifesci hcls
Subject: Re: Seeking Help with finding an assertion

Hi Chris,

Thanks for pointing out the potential flaws of their method. It sounded
like there is room for improvement in terms of the accuracy of database
contents and the method of assessing database accuracy. Don't get me
wrong. I think highly of GO. :-)

I'm also thinking more about what "negative knowledge" really means.
Does it mean any or all of the following:

1. inconsistent knowledge
2. inaccurate knowledge
3. incomplete knowledge
4. knowledge with uncertainties

Can SW/ontologies help turn "negative knowledge" to "positive
knowledge"?

-Kei

Chris Mungall wrote:

>
>
> On Jul 4, 2007, at 8:27 PM, Kei Cheung wrote:
>
>>
>> As a follow-up example, a study for estimating the error rate of
>> Gene Ontology (GO) was done:
>>
>> http://www.pubmedcentral.nih.gov/articlerender.fcgi?
>> artid=1892569#id2674403
>>
>> The study showed that the GO term annotation error rate estimates
>> for the GoSeqLite database were found to be 13% to 18% for curated
>> non-ISS annotations, 49% for ISS annotations, and 28% to 30% for  all

>> curated annotations. (ISS stands for inferred from sequence
>> similiarity). Despite these findings, the authors concluded that GO
>> is a comparatively high quality source of informaton. Integration  of

>> databases involving significant error rates, however, can impact
>> negatively the quality of science.
>
>
> I have not yet properly digested this paper, but on a cursory reading

> there appear to be a few serious flaws. First, a lack of
> understanding of basic ontology principles - annotations to less
> specific classes in the graph are treated as errors. Second, the
> authors appear to make a lot of incorrect assumptions about how ISS
> annotations are curated.
>
> It's curious they predict such a high error rate yet don't provide
> any examples.
>
>>
>> -Kei
>>
>> Kei Cheung wrote:
>>
>>>
>>> Hi Karen,
>>>
>>> Your questions remind me of the following classic article written
>>> by Robert Robbins on "Challenges in the Human Genome Project".
>>>
>>> http://www.esp.org/umdnj.pdf
>>>
>>> Although it doesn't directly answer the questions, in the
>>> "Nomenclature Problems" section (p. 20-21), it discusses the
>>> significant problem of inconsistent knowledge representation. It
>>> says that it's mistake to believe  that terminology fluidity is  not

>>> an issue biological in database design. It also says that many
>>> biologists don't realize that, in a database bulit with 5% error  in

>>> the definition of individual concepts, a query that joins  across 15

>>> concepts has less than 50% chance of returning an  adequate answer.
>>> The section also points out the importance of  formal representation

>>> of scientific knowledge in addressing the  inconsistency and
>>> nomenclature problems. Semantic Web and standard  ontologies provide

>>> a solution to these database problems. We just  don't simply convert

>>> an existing database syntactically into a  semantic web format, but
>>> we also need to do careful semantic  conversion to eliminate as many

>>> errors, ambiguities, and  inconsistencies as possible in order to
>>> reduce the costs of  knowledge retrieval and discovery.
>>>
>>> -Kei
>>>
>>> Skinner, Karen (NIH/NIDA) [E] wrote:
>>>
>>>> Recently I read somewhere (on this list, a blog, a news story,
>>>> where...?) an assertion that struck me as an interesting passing
>>>> fact at the time.   As I recall, it indicated that more websites
>>>> are accessed via a search engine than by typing a URL into a
>>>> browser web address bar.
>>>>
>>>> Alas, I did not save the reference, and now I am looking for the
>>>> proverbial needle in a haystack. Namely, what is the exact
>>>> assertion, who asserted it, and where did they make it?  If  anyone

>>>> in the world has this information or knows how to get it,  or or
>>>> has related data, I imagine they would belong to this list.  I
>>>> would be most grateful for any useful pointer.
>>>>
>>>> Along this same vein, if anyone has any statistics, data,
>>>> anecodotes or information related to the cost of
>>>> (1) "friction" arising from inefficient or inappropriate efforts
>>>> at information retrieval
>>>> and
>>>> (2) the cost of "negative knowledge" about an existing resource  or

>>>> data,
>>>>
>>>> these, too, would be helpful.
>>>>
>>>> (For example, with respect to #2 above, we are all familiar with
>>>> comparison shopping for goods and services. We seek data/
>>>> information about prices and quality , but at what point does the
>>>> expenditure of that effort exceed the value of the information
>>>> learned?)
>>>>
>>>> I am not looking for examples at the level of a philosophy or
>>>> ecnomics Ph.D. thesis, but rather a few examples in the sciences
>>>> that can be used at the level of an "elevator speech."
>>>>
>>>>
>>>> Karen Skinner
>>>> Deputy Director for Science and Technology Development
>>>> Division of Basic Neuroscience and Behavior Research
>>>> National Institute on Drug Abuse/NIH
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>
>>
>>
>>
>
>

Received on Friday, 6 July 2007 15:07:50 UTC