Re: Towards ontology metrics to guide the development and assessment of "good" ontologies

Hi Michel,

Thanks for bringing up this topic.  As you know, it's something I've  
been thinking about for some time.   When I was at MITRE, I was tasked  
with proposing a research program in on ontology evaluation methods  
and metrics and came up with an approach that NIST liked and was  
intending to support, however between federal budgets and other  
bureaucratic concerns, etc, it all got put on hold.  Fortunately, I  
was able to presented these ideas at the Infectious Disease Ontology  
workshop and at the EBI Industry Program and refer you to the slides  
for further discussion.

The slides are here:  http://ontology.buffalo.edu/08/IDO/Luciano.pdf

When I was at Manchester I ran this by Bijan Parsia for ontology  
evaluation and suggested that the approach could also be used for  
ontology engineering.  He liked that idea and thought I should propose  
it as a development methodology.  The idea is to build ontologies in  
what I termed "semantic components" which could be independently  
constructed and evaluated and reused.  Circumstances prevented me from  
then further writing it up, but times have changed and this is back on  
my agenda when I return from summer holiday.

The approach I came up with starts with the assumption that it only  
makes sense to evaluate an ontology within the context of a use.  Is  
and ontology good?  Good for what? What is the intended use of the  
ontology.  It then separates the function of the ontology from the  
metric used to measure of the ontology.  I think for this I was  
influenced by a paper by Bernhard Riemann, (have to check when I'm  
back in Boston) coming from the idea that what is being measured is  
independent of the measurement. For example, the measurement of length  
of an object is independent from the object being measured. The second  
part breaks the function down into three levels, function,  
requirements of the design, and the semantic components needed to meet  
the requirements and perform the function.  Each of these three  
functional levels is and can be evaluated independently (which  
supports not only modular construction of ontologies, but modular  
testing and  reuse of components).

Note that part of what lies ahead for us (the community) is a research  
project -- some metrics are and have been established from our (the  
ontology community's) ancestry in natural language processing and  
software development. Others we have to create for our purpose.

Hope this helps the discussion.

Cheers,
Joanne


On Jul 27, 2010, at 12:16 PM, Michel_Dumontier wrote:

> There are two issues I'd like to raise to our communities:
>
> 1 - What metrics should we use to assess ontology quality and  
> determine whether an ontology is "good".
> 2 - Can we re-factor existing, and well used terminologies (SNOMED- 
> CT, NCI-T, ICD) into "good" ontologies for health care and the life  
> sciences?
>
> m.
>
>> -----Original Message-----
>> From: Bill Hogan [mailto:hoganwr@gmail.com]
>> Sent: Tuesday, July 27, 2010 11:57 AM
>> To: obo-discuss@lists.sourceforge.net
>> Subject: Re: [Obo-discuss] Ontological Realism and OBO Foundry  
>> Criteria
>>
>> We don't have _good_ ontologies for basic electronic medical record
>> "stuff", like diseases, hypersensitivity conditions (although I  
>> made a
>> start), medications, procedures, laboratory tests (again, work  
>> started
>> but still woefully incomplete, even without hitting any hard cases),
>> etc.
>>
>>
>> Bill
>>
>
>

Received on Tuesday, 27 July 2010 17:37:08 UTC