W3C home > Mailing lists > Public > public-xg-mmsem@w3.org > November 2006

[MMSEM-UC] review of MPEG-7 metadata interoperability use case

From: Suzanne Little <Suzanne.Little@isti.cnr.it>
Date: Tue, 14 Nov 2006 18:32:17 +0100
To: dasiop@iti.gr, tzouvaras@image.ntua.gr, michael.hausenblas@joanneum.at, public-xg-mmsem@w3.org
Message-id: <4559FDA1.1070707@isti.cnr.it>

Dear all,

Please find below my comments on the MPEG-7 metadata interoperability 
use case as per action 12 from the last teleconference.


Review of Use Case: MPEG-7 metadata interoperability from W3C Multimedia 
Semantics Incubator Group

(using version last edited 2006-10-26 10:12:15 by RaphaelTroncy)

Overall, I think it's a good use case: relevant to the XG and 
well-expressed. I'll be interested to read the solutions that are yet to 
be fully described in section 3.

1. Introduction:
A minor typographical comment about the first point -- the captions 
underneath the example fragments in the introduction section were a bit 
confusing. Initially I thought they were headings. Perhaps some 
adjustment of the layout could clear that up -- caption inside the 
border, smaller text?

The second point is interesting but I'm not sure if I've understood 
correctly. :-) What I think you're saying is:

Desired behaviour: The query "Images that depict Zidane" should return 
not only images that are directly described but also images that have 
segments (StillRegion) that are described as depicting Zidane. I assume 
that you want such a query to return the containing images and not the 

Problem: Because MPEG-7 doesn't formally define decomposition 
relationships, you can't write a (single? simple? distributed?) semantic 
query that retrieves images (not segments) in both of these cases.

The overall question is: Does this problem (that MPEG-7 doesn't formally 
define decomposition relationships) restrict the interoperability of 
multimedia descriptions that use MPEG-7? I think you address this 
partially in section 2.1 when you mention that the MPEG-7 ontology and 
the upper ontology provides these relationships and therefore enables 
this type of query. So a related question is: if different MPEG-7 or 
multimedia ontologies record these relationships in different ways 
(e.g., subClass vs. partOf) then does this restrict interoperability?

I agree with Raphael's comment in principle although I'm not sure if 
everything in MPEG-7 is a segment. Bearing in mind that my understanding 
is highly influenced by Jane's ontology which is what I've had the most 
exposure to: StillRegion is a subClass of both Segment and Image which 
are both subClasses of MultimediaContent. This provides a formal 
relationship for decomposition based on the class hierarchy. I don't 
know how other MPEG-7 ontologies have addressed this.

Some thoughts: In Jane's MPEG-7 ontology the subClass relationship is 
used to imply the decomposition relationships. However, when the ABC 
model is used as an upper ontology mpeg7:MultimediaContent is a subClass 
of abc:Manifestation which has the properties abc:partof/abc:contains 
abc:Entity. This could be used to make explicit decomposition 
relationships. E.g., mpeg7:StillRegion abc:partof mpeg7:Image; 
mpeg7:Image abc:contains mpeg7:StillRegion etc. Using the upper ontology 
to provide the decomposition relationship could make the modelling more 
interoperable if different multimedia ontologies are used?

2. Existing MPEG-7 Ontologies

2.1  Using the MPEG-7/ABC Ontology

I like the way you've expressed the integration of MPEG-7, ABC and the 
soccer ontology. I think it's a nice simple example that shows some of 
the advantages. It might be worth adding a couple of references (see 
References at the end).

One point that isn't made explicitly is how the use of an upper ontology 
to relate the domain ontology to the media ontology promotes 
interoperability. For example, say I also have a collection of football 
media which I have annotated in a similar way but using my sports 
ontology since, as an Australian, I follow 4 codes of football. You 
would be able to construct a query (e.g., mpeg7:Image depicts abc:Event 
and abc:Event abc:hasAction abc:Action and abc:Action abc:hasParticipant 
"Zidane") that would operate successfully over both our collections 
regardless of the specific annotation we've used. It's not perfect 
(e.g., if I had images of a Rugby player scoring who is called "Zidane" 
they would also be returned or I wanted to make a query such as "show me 
all soccer images" since I need to explicitly relate the concepts from 
the domain ontologies) but it does enable some distributed querying and 
provides an integration framework for domain ontologies. I think this is 
probably the key advantage of this type of approach as compared with 
using an MPEG-7 ontology as a core ontology (as described in 2.2).

2.2 Using the MPEG-7/Tsinaraki Ontology

I'm not familiar with this ontology although the approach sounds 
interesting. Why wouldn't it be possible to map this ontology onto an 
upper ontology such as ABC and thereby integrate it with the model in 2.1?

3. Possible Solutions

I'm interested to see what you propose as solutions particularly for 
your first point. How to map between the chosen syntactic structure and 
the semantics in the ontology is an important question. It's hard to 
comment specifically without more detail. I'm particularly curious about 
the first proposed solution. Do you have some examples? Are you 
referring to Jane's work integrating CIDOC/DC/MPEG7/ABC etc? What kinds 
of relationships other than the equality and subClass/Property provided 
by OWL will be required?

Some references

http://metadata.net/mpeg7/ - the latest mpeg7.owl plus links to some 
relevant publications
http://jodi.ecs.soton.ac.uk/Articles/v02/i02/Lagoze/ - C. Lagoze, J. 
Hunter, "The ABC Ontology and Model (v3.0)", Journal of Digital 
Information, Vol 2 , Issue 2, November 2001
http://www.itee.uq.edu.au/~jane/jane-hunter/events/final_paper.pdf - J. 
Hunter, "Enhancing the Semantic Interoperability of Multimedia through a 
Core Ontology", IEEE Transactions on Circuits and Systems for Video 
Technology, Special Issue on Conceptual and Dynamical Aspects of 
Multimedia Content Description, Feb 2003
Received on Tuesday, 14 November 2006 20:35:13 GMT

This archive was generated by hypermail 2.2.0+W3C-0.50 : Tuesday, 8 January 2008 14:21:20 GMT