API for Media Resource 1.0

Editors' copy $Date: 2010/01/18 12:59:02 $ @@ @@@@ @@@@

This version:
mediaont-api-1.0.html
Latest version:
http://dev.w3.org/2008/video/mediaann/mediaont-api-1.0/mediaont-api-1.0.html?content-type=text/html; charset=utf-8
Editors:
WonSuk Lee, Electronics and Telecommunications Research Institute (ETRI)
Florian Stegmaier, University of Passau
Chris Poppe, Ghent University

Abstract

This specification defines a client-side API to access metadata information related to media resources on the Web. The overall purpose of the API is to provide developers with a convenient access to metadata information stored in different metadata formats. The API will be introduced in an abstract manner using the interface definition language Web IDL. Thereby, the Media Ontology Core Properties will be used as a pivot vocabulary in the API.

Status of this Document

This document is an editors' copy that has no official standing.

Table of Contents

1 Introduction
    1.1 Formats in scope
    1.2 Formats out of scope
    1.3 Terminology
2 API Description
    2.1 Design consideration
    2.2 API for identifying the media resource and metadata
    2.3 API for accessing properties
        2.3.1 Identification Properties
            2.3.1.1 Identifier
            2.3.1.2 Title
            2.3.1.3 Language
            2.3.1.4 Locator
        2.3.2 Creation Properties
            2.3.2.1 Contributor
            2.3.2.2 Creator
            2.3.2.3 CreateDate
            2.3.2.4 Location
        2.3.3 Content Properties
            2.3.3.1 Description
            2.3.3.2 Keyword
            2.3.3.3 Genre
            2.3.3.4 Rating
        2.3.4 Relational Properties
            2.3.4.1 Relation
            2.3.4.2 Collection
        2.3.5 Rights Properties
            2.3.5.1 Copyright
            2.3.5.2 Policy
        2.3.6 Distribution Properties
            2.3.6.1 Publisher
            2.3.6.2 TargetAudience
        2.3.7 Fragments Properties
            2.3.7.1 Fragments
            2.3.7.2 NamedFragments
        2.3.8 API for Technical Properties
            2.3.8.1 FrameSize
            2.3.8.2 Compression
            2.3.8.3 Duration
            2.3.8.4 Format
            2.3.8.5 Samplingrate
            2.3.8.6 Framerate
            2.3.8.7 Bitrate
            2.3.8.8 NumTracks
    2.4 API for retrieval of reason of an error
    2.5 API for iterating
        2.5.1 getPropertyNamesWithValues
        2.5.2 getSourceFormatsWithValues
        2.5.3 getOriginalData
3 Examples of how to use the API
4 Security Considerations

Appendices

A Web IDL description for API
B References(Normative)
C References(Non-Normative)
D Acknowledgements (Non-Normative)


1 Introduction

This section is informative.

Editorial note 
This part will be elaborated in the perspectives of reason why we want to provide an API and an explanation on who should use/implement it.

This specification defines a client-side API to access metadata information related to media resources on the Web. The overall purpose of the API is to provide developers with a convenient access to metadata information stored in different metadata formats. Thereby, the Media Ontology Core Properties will be used as a pivot vocabulary in the API. The description of relations between these core properties and the metadata formats in scope (1.1 Formats in scope) are stored in the Media Ontology in order to provide cross-community data integration. The API will be introduced in an abstract manner using the interface definition language Web IDL. The decision to use Web IDL, which offers bindings for ECMAScript and Java, can be based on the Use Cases and Requirements for Ontology and API for Media Object 1.0. This document clearly states that the focus for this API lies on multimedia services on the Web.

The API serves as a mediator between a developer and the underlying Ontology for Media Resource 1.0 with the goal to support interoperability between metadata formats. It offers GET operations to retrieve particular metadata informations represented in a certain metadata format related to media ressources on the Web.

The initial version of this document contains only a limited description of API. In addtion, there are many open issues including definitions of return types. Nevertheless it is being published with the aspiration to gather wide feedback on the yet available API design.

2 API Description

Editorial note 
This part will be elaborated with a precise description of the actual methods in the API, (so these are the methods that can be found in the Strawman API design and note), described using Web IDL. This section needs to define and explain all aspects of those methods (parameters and return values).
Editorial note 
There are many open issues about SET interfaces, so these issues will be covered later.

2.1 Design consideration

This part defines the interface for accessing the metadata. The API has been described using Web IDL. All properties can be accessed through a specific operation, encapsulated in the MediaResource interface within the mawg module. Currently, only read access is defined. When an attempt to read a property fails, diagnostics information can be obtained using a diagnosis operation. Certain properties have complex structures so the return type consists of one or more object types. The structure of these objects is still to be defined, currently these follow different interfaces which have been defined in the returnValues module.

Note that, some of these interfaces inherit from the MAObject interface and the Language interface.

Web IDL syntax:
     	
		interface MAObject {
			attribute DOMString unstructuredValue;
	  		attribute DOMString uri;
			attribute DOMString sourceFormat;
			attribute DOMString fragmentIdentifier;
			attribute DOMString mappingType;
	  	};
		
		interface Language {
			attribute DOMString language;
			};
			
							
Description:

The MAObject interface has an attribute unstructeredValue using DOMString. If it is not clear how a certain value for a property should be structured, this attribute allows to describe the value in plain text. A uri attribute is provided which should hold an URI. This can for example be used to represent labels as uri's. A language attribute allows to specify the language of the returned property. The sourceFormat attribute allows to specify the metadata source from which the metadata was retrieved. The fragmentIdentifier attribute determins the fragment for which the metadata is relevant. Lastly, the mappingType attribute specifies the kind of mapping as discussed in the semantic level mappings.

The Language interface includes an attribute language that holds the language of the metadata.

We consider two scenarios where the API could be implemented: either in the user agent (scenario 1) or as a web service (scenario 2). The two scenarios are shown in the figure.

User agent

The API is implemented in the user agent (e.g., browser or browser plugin) and exposed as a JavaScript API (using the WebIDL JavaScript binding). The user agent includes the components for metadata access (possibly extraction) and mappings for a supported set of formats. The metadata sources (the media resource and/or metadata document(s)) must be retrievable and access (establish connection, retrieval) of the metadata sources is handled by the user agent.

Web service

The API is implemented as a Web service. [Editorial note: there is no appropriate WebIDL binding (such as WSDL), thus appropriate formats for method calls (e.g. HTTP query parameters) and return values (XML, Jason) need to be defined] Such an implementation would be typically used by a non-UI client, such as an agent harvesting metadata. However, the API could be also accessed from a user agent, and used the same way as described in scenario 1 by the help of a JavaScript library for accessing the web service. At the back-end of the web service, this scenario also allows supporting a media repository (e.g. content provider's archive database, movie store) from which the user agent could directly retrieve metadata sources and which might have a custom metadata format not supported by a user agent. In contrast to an integrated component (see scenario 1), an implementation of the API in a web service could do more complex mappings on the fly as a component integrated in a user agent, and can be more flexible (e.g., supporting additional formats).

Overview of different API options.

In both scenarios, the access to the metadata properties needs the following stack of components:

  • An implementation of the API for Media Resource (as defined in this document), which providers the actual getter methods for the properties.

  • An implementation of the mappings from a specific source format to the properties of the media ontology (as defined in Ontology for Media Resource 1.0).

  • A format specific API to access the metadata. This is can be an API for accessing a metadata document describing a media resource (e.g. an XML parser and a set of XPath statements) or an extractor the read metadata embedded in the media resource (e.g. a library to read EXIF information from JPEG images). In order to define the context on which the API for Media Resource is working (cf. Section 2.2), it is assumed that there is at least a unidirectional reference from the media resource to the metadata document or vice versa. If this is not the case such a reference needs to be provided by the web application (scenario 1), web service (scenario 2) or media repository (scenario 2).

2.2 API for identifying the media resource and metadata

This part defines part of the interface for identifying the Media Resource for which the API will grant access to. One general operation is defined that allows access to the properties. The specific property is passed as an argument and a list of objects is returned that hold the values according to the requested property. Depending on the requested property, the returned objects follow a different interface. These are listed in the following sections.

Web IDL syntax:
Description:

The setContext operation allows to set the specific Media Resource and metadata sources for which the API applies. The mediaResource argument identifies the Media Resource. The implementation of the API should try to find relevant metadata sources for this Media Resource. Additionally, the metadataSources argument allows to identify other metadata sources which should be included by the API. This argument is an array of objects, each implementing the MetadataSource interface. This interface holds an uri identifying the metadata source (metadataSource) and the name of the actual metadata format (sourceFormat). The return value of the operation is a boolean stating whether the Media Resource and metadata sources were succesfully loaded.

Return type:

A boolean stating whether the Media Resource and metadata resources could be identified.

2.3 API for accessing properties

This part defines part of the interface for accessing the metadata that describes a Media Resource. One general operation is defined that allows access to the properties. The specific property is passed as an argument and a list of objects is returned that hold the values according to the requested property. Depending on the requested property, the returned objects follow a different interface. These are listed in the following sections.

Web IDL syntax:
Description:

The getProperty operation allows to retrieve the value of a certain property. The propertyName argument identifies the property for which the values need to be retrieved. Optional arguments allow to refine the request. The sourceFormat identifies a specific metadata format. If a metadata format is defined, only the metadata available in the corresponding metadata format are retrieved. The subtype argument identifies a subtype. Some properties can be filtered on subtypes. The language argument allows to identify the language of the metadata. Only if the metadata is available in the specified language, the values are returned. Finally, the fragment argument allows to identify the specific media fragment for which the metadata is requested.

Return type:

An array of objects that hold the values of the requested property.Depending on the requested property, these objects implement a different interface. The next sections list the different properties and the returned objects.

2.3.1 Identification Properties

2.3.1.3 Language
Description:

The language property specifies a language used in the entity, Recommended best practice is to use BCP 47 [BCP 47].

Return type:

An array of object elements implementing the StringObject interface. The value attribute should represent the language as a plain string.

interface StringObject: MAObject {
			attribute DOMString value;
		};
					
Example:

N/A

2.3.2 Creation Properties

2.3.3 Content Properties

2.3.8 API for Technical Properties

This part defines the interfaces for accessing the technical metadata that describe information for dealing with the creation or storage encoding processes or formats of the resource.

2.5 API for iterating

This part defines part of the interface for iterating over the available metadata for a Media Resource.

3 Examples of how to use the API

Editorial note 
This part will illustrate how to use the API in the actual (working) implementations

4 Security Considerations

This section is informative.

Editorial note 
It is required to be considered in the perspectives of developer, user and content’s provider. This will be revised with more security issues.

This specification defines a client-side API to access metadata information related to media resources on the Web. These APIs will provide the methods for getting and setting metadata information which can be in one of different formats, either as separate document or embedded in media resources. So far concerning the setting method, there could be security consideration issues.

There are related activity and technical documents in W3C such as Policy Requirements in DAP WG, ODRL 1.1, P3P 1.1 and PLING Wiki.

A Web IDL description for API

module mawg {
	interface MediaResource {

	  	//Media Resource context
	  	boolean setContext(in DOMString mediaResource, in optional Object[] metadataSources );
	  	
	  	// Property Access
	  	object[] getProperty(in DOMString propertyName, in optional DOMString sourceFormat, in optional DOMString subtype,
	  			       in optional DOMString language, in optional DOMString fragment );
	  	
	  	//Iterating
	  	DOMString[] getPropertyNamesWithValues(in optional DOMString sourceFormat,
	  						      in optional DOMString language, in optional DOMString fragment);
	  	DOMString[] getSourceFormatsWithValues(in optional DOMString language);
	  	DOMString getOriginalData(in DOMString sourceFormat);
	  	
	  	//Operation for retrieval of the reason of an error
	  	DOMString getDiagnosis();
	  	
	  };
  	
	interface MetadataSource {
  		attribute DOMString metadataSource;
  		attribute DOMString sourceFormat;
  	};
	  	
	module returnValues {
	
		interface MAObject {
			attribute DOMString unstructuredValue;
	  		attribute DOMString uri;
			attribute DOMString sourceFormat;
			attribute DOMString fragmentIdentifier;
	  		attribute DOMString mappingType;
	  	};
		
		interface Language {
			attribute DOMString language;
			};
		
			
		interface StringObject: MAObject, Language {
			attribute DOMString value;
	  	};
	  	
	  	interface UnsignedLongObject: MAObject {
			attribute unsigned long value;
	  	};
	  	
	  	interface UnsignedShortObject: MAObject {
			attribute unsigned short value;
	  	};
	  	
	  	interface FloatObject: MAObject {
			attribute float value;
	  	};
	  	
	  	interface Identifier: MAObject {
	  		attribute DOMString value;
	  		attribute DOMString type;
	  	};
	  	
	  	interface Title: MAObject, Language {
	  		attribute DOMString value;
	  		attribute DOMString type;
	  	};
	  	
	  	interface Contributor: MAObject {
			attribute DOMString id;
			attribute DOMString role;
		};

	  	interface Date: MAObject {
		  	attribute DOMString date;
		  	attribute DOMString type;
	  	};		
	  	
	  	interface Location: MAObject, Language {
		  	attribute DOMString name;
		  	attribute Float longitude;
		  	attribute Float latitude;
		  	attribute Float altitiude;
		  	attribute DOMString system;
	  	};
	  	
	  	interface Rating: MAObject, Language {
	               	 attribute DOMString issuer;
			attribute short value;
			attribute short minimum;
			attribute short maximum;
			attribute DOMString context;
	  		attribute DOMString type;
		};

	  	interface Relation: MAObject, Language {
			attribute DOMString id;
			attribute DOMString relationship;
		};
	  	
	  	interface Copyright: MAObject, Language {
			attribute DOMString statement;
			attribute DOMString[] holder;
		};

		interface Policy: MAObject, Language {
			attribute DOMString statement;
			attribute DOMString organization;
			attribute DOMString type;
		};
	  	
	  	interface TargetAudience: MAObject, Language {
			attribute DOMString issuer;
			attribute DOMString classification;
		};
	  	
	  	interface Fragment: MAObject, Language {
			  attribute DOMString role;
			  attribute DOMString identifier;
		};
		
		interface NamedFragment: MAObject, Language {
			  attribute DOMString name;	
			  attribute DOMString identifier;
		};
		
	  	interface FrameSize: MAObject {
			  attribute unsigned long width;
			  attribute unsigned long height;
		};
	};
};

	  

B References(Normative)

[Ontology for Media Resource 1.0]
Ontology for Media Resource 1.0, WonSuk Lee, Tobias Bürger, Felix Sasaki, Véronique Malaisé, Florian Stegmaier and Joakim Söderberg. W3C Working Draft, 18 June 2009. Available at http://www.w3.org/TR/mediaont-10/ .

C References(Non-Normative)

[BCP 47]
BCP 47 (Matching of Language Tags, Tags for Identifying Languages), A. Phillips and M. Davis, Editors. Available at http://www.rfc-editor.org/rfc/bcp/bcp47.txt.
[Policy Requirements]
Policy Requirement in DAP WG. Available at http://dev.w3.org/2009/dap/policy-reqs/.
[ODRL 1.1]
Open Digital Rights Language (ODRL) Version 1.1, Renato Iannella. W3C Note, 19 September 2002. Available at http://www.w3.org/TR/odrl/.
[P3P 1.1]
The Platform for Privacy Preferences 1.1 (P3P1.1) Specification, Rigo Wenning and Rigo Wenning. W3C Working Group Note, 13 November 2006. Available at http://www.w3.org/TR/P3P11/.
[PLING Wiki]
PLING Wiki. Available at http://www.w3.org/Policy/pling/.
[Web IDL]
Web IDL, Cameron McCormack, Editor. W3C Working Draft, 19 December 2008. Available at http://www.w3.org/TR/WebIDL/ .

D Acknowledgements (Non-Normative)

This document is the work of the W3C Media Annotations Working Group.

Members of the Working Group are (at the time of writing, and by alphabetical order): Werner Bailer (JOANNEUM RESEARCH), Tobias Bürger (University of Innsbruck), Eric Carlson (Apple, Inc.), Pierre-Antoine Champin ((public) Invited expert), Ashish Chawla ((public) Invited expert), Jaime Delgado (Universitat Politècnica de Catalunya), Jean-Pierre EVAIN ((public) Invited expert), Philip Jägenstedt (Opera Software), Ralf Klamma ((public) Invited expert), WonSuk Lee (Electronics and Telecommunications Research Institute (ETRI)), Véronique Malaisé (Vrije Universiteit), Erik Mannens (IBBT), Hui Miao (Samsung Electronics Co., Ltd.), Thierry Michel (W3C/ERCIM), Frank Nack (University of Amsterdam), Soohong Daniel Park (Samsung Electronics Co., Ltd.), Silvia Pfeiffer (W3C Invited Experts), Chris Poppe (IBBT), Víctor Rodríguez (Universitat Politècnica de Catalunya), Felix Sasaki (Potsdam University of Applied Sciences), David Singer (Apple, Inc.), Florian Stegmaier ((public) Invited expert), John Strassner ((public) Invited expert), Joakim Söderberg (ERICSSON), Thai Wey Then (Apple, Inc.), Ruben Tous (Universitat Politècnica de Catalunya), Raphaël Troncy (CWI), Vassilis Tzouvaras (K-Space), Davy Van Deursen (IBBT).

The people who have contributed to discussions on public-media-annotation@w3.org are also gratefully acknowledged.