This specification defines an API to provide webapps with access to media files, such as images, videos or music, and their metadata in the device.

Note that this specification is an unofficial draft and that the W3C System Applications WG at this moment is not taking up this work.

Introduction

This API provides webapps with access to media files in the device through a MediaDB interface for each media type, images, videos or music. Each media file is represented by a MediaDBRecord, which includes name, size, type, modification date and metadata (image thumbnails, song titles, etc) of the stored files.

Compared to directly using the W3C file API's for in media handling webapps this API provides a simpler interface and better performance.

The specification is partly inspired by Firefox OS MediaDB API, which is a library that integrates both the Firefox OS DeviceStorage API and the W3C IndexedDB API into a single API. However, the ambition with this specification is to define an API that is independent on any other underlying API, except the W3C File API [[!FILE-API]]), that is subject for W3C standardization and that could be implemented in any web user agent.

Use cases

Examples of webapps that using this API are:

Summary of the functionality of this API

This specification defines conformance criteria that apply to a single product: the user agent that implements the interfaces that it contains.

Implementations that use ECMAScript to implement the APIs defined in this specification MUST implement them in a manner consistent with the ECMAScript Bindings defined in the Web IDL specification [[!WEBIDL]], as this specification uses that specification and terminology.

Terminology

The term webapp refers to a Web application, i.e. an application implemented using Web technologies, and executing within the context of a Web user agent, e.g. a Web browser or other Web runtime environment.

The Promise interface provides asynchronous access to the result of an operation that is ongoing, has yet to start, or has completed, as defined in [[!ES6]].

The EventHandler interface represents a callback used for event handlers as defined in [[!HTML5]].

The concepts queue a task and fire an event are defined in [[!HTML5]].

The terms event handler and event handler event types are defined in [[!HTML5]].

The File interface is defined in [[!FILE-API]].

The Blob interface is defined in [[!FILE-API]].

Security and privacy considerations

This is a security and privacy sensitive API and a webapp must have permission to use the API. Permission to access a media database for a media type is requested through the MediaStorage.get method. The manner in which permission is given or not varies depending on the type of web runtime environment in which this API is implemented and could be based on:

The manner in which permission to use this API differs depending on the type of web runtime it is implemented in. For example, a web runtime for secure installed web applications may be able to open up this API so that no explicit user content is needed, while an implementation in a web browser may use a combination of web security mechanisms, such as secure transport (https:), content security policies (CSP) and a signed manifest, and user consent to open up the API.

The Permissions API, which currently is an unofficial draft, is an API that allows a web application to be aware of the status of a given permission, to know whether it is granted, denied or if the user will be asked whether the permission should be granted or not. If this API gets consensus and becomes a W3C standard, and also is extended to support permission requests, it may be possible to use his API for permission to the MediaStorage API.

W3C has activities on Trust and permissions. For example, see Workshop on trust and permissions for Web applications 3–4 September 2014, Paris, France and the proposed Community Group as well as Requirements for Powerful Features.

Navigator Interface

readonly attribute MediaStorage mediaStorage
The object that exposes the Media Storage API.

MediaStorage Interface

This interface exposes the get() method to retrive the media database for a type of media (images, videos or audio).

Promise<MediaDB> get ()
Get a media database for a media type. Depending on platform the user agent MAY provide a user dialogue allowing the user to select the media directories for which the client webapp should be given access to.

This method MUST return a Promise and then run the following steps asynchronously:

  1. Create a new Promise, getMediaStoragePromise, return it and run the remaining steps asynchronously.
  2. Get the MediaDB object for the specified mediaType reflecting the selected media storage areas. Note that the user agent MAY provide a user dialogue for selection of storage area and/or sub directories.
  3. If no media database for the selected media type could be retrieved, no user permission is given or the operation failed due to any other reason then reject getMediaStoragePromise with DOMException TBD and abort the remaining steps.
  4. If the MediaDB object was successfully retrieved resolve getMediaStoragePromise with the MediaDB object as argument.

MediaType mediaType
The type of media.

Selection of media storage areas and user permission may have to be further investigated. See for example Chrome mediaGalleries getMediaFileSystems and addUserselectedFolder and Tizen Content API getDirectories.

Interface MediaDB

This interface defines an object that represents the media storage for either "images", "videos" or "audio".

readonly attribute MediaDBState state
The state of the MediaDB object.
attribute EventHandler onready
Event handler that corresponds to the ready event, which is fired when the MediaDB object is ready to use.
attribute EventHandler onunavailable
Event handler that corresponds to the unavailable event, which is fired when the when the media storage becomes unavailable. The MediaDB state attribute states the detailed reason for the unavailable MediaDB. It is one the values nocard or unmounted.
attribute EventHandler oncreated
Event handler that corresponds to the created event, which is fired when one or more files were created.
attribute EventHandler ondeleted
Event handler that corresponds to the deleted event, which is fired when one or more files were deleted.
attribute EventHandler onscanstart
Event handler that corresponds to the scanstart event, which is fired when scanning for new and deleted files starts.
attribute EventHandler onscanend
Event handler that corresponds to the scanend event, which is fired when scanning for new and deleted files ends.
attribute EventHandler onpartlyunavailable
Event handler that corresponds to the partlyunavailable event, which is fired when a storage area becames permanently unavailable but one or more other storage areas are still available. This happens for example when an SD-card is removed but the internal memory is still available.
Promise<void> enumerate ()
The method gets MediaDBRecord objects and passes them to the specified callback. By default, records are returned in alphabetical order by name, but optional arguments allow you to specify a sort direction and filering criteria. The returned Promise is fulfilled with no argument when the enumeration process has been completed and rejected when an error occurs.
EnumerateCB callback
The callback is called once for each entry in the media storage.
optional EnumerateOptions options
The options for the enumerate() method. All fields are optional.
Promise<void> scan ()
Initiates scanning for new and deleted files. If the scan finds new files, it reports them with one or more created events. If the scan finds that files have been deleted, it reports them with one or more deleted events. Scanning is performed continuously in the background but the frequency of the scanning is platform dependent. Client webapps could use this method to assure that the media database gets updated.

This method must return a Promise and then run the following steps asynchronously:

  1. Create a new Promise, scanPromise, return it and run the remaining steps asynchronously.
  2. If scanning is not already in progress start to scan the media content directories associated with this MediaDB object for new and deleted files and queue a task to fire an event named scanstart at the MediaDB object.
  3. If scanning fails by any reason reject scanPromise with DOMException TBD, abort the remaining steps and fire an event named scanend at the MediaDB object .
  4. If the scan finds new files, it reports them to the webapp with one or more created events. If the scan finds that files have been deleted, it reports them with one or more deleted events.
  5. When scanning has been completed resolve scanPromise with no argument and fire an event named scanend at the MediaDB object.

It must be further investigated if this method is needed. Normally the UA performs scanning in the background. However, for certain big storage areas it might be needed to have a user/application controlled start of scanning. The need for the scanstart and scanend events must also be investigated.

Promise<MediaDBRecord> get ()
Given a filename this method looks up the named file and passes the MediaDBRecord object with the resolved Promise.

This method must return a Promise and then run the following steps asynchronously:

  1. Create a new Promise, getFileInfoPromise, return it and run the remaining steps asynchronously.
  2. Get the MediaDBRecord object for the specified file from the storage.
  3. If the file can not be found or any other error occurrs reject getFileInfoPromise with DOMException TBD and abort the remaining steps.
  4. If the MediaDBRecord object was successfully found resolve getFileInfoPromise with the MediaDBRecord object as argument.

DomString fileName
The name of the file to get.
Promise<void> updateMetadata ()
Updates the metadata of a given file in the storage and the metadata of the corresponding MediaDBRecord.

Needs to be investigated. Should we have an argument stating if the metadata of only the MediaDBRecord should be updated or if both the MediaDBRecord and the file itsel should be updated? What happens if two applications simultaneously changes the metadata of a file?

. Note that FFOS MediaDB does only allows the update of metadata in the IDB, not for the file itself. The Tizen Content API (see https://developer.tizen.org/documentation/dev-guide/2.2.1?redirect=https://developer.tizen.org/dev-guide/2.2.1/org.tizen.web.appprogramming/html/api_reference/api_reference.htm, Web Device API reference, Content API, interface Content, defines an array of "editableAttributes" but the metadata of the file can not be updated. See the update() method. The Chrome media galleries API, https://developer.chrome.com/apps/mediaGalleries, does not provide a method for updating metadata.

This method must return a Promise and then run the following steps asynchronously:

  1. Create a new Promise, updateMetadataPromise, return it and run the remaining steps asynchronously.
  2. Get the MediaDBRecord object and the File object for the specified file from the storage.
  3. If the record and/or file can not be found or any other error occurrs reject updateMetadataPromise with DOMException TBD and abort the remaining steps.
  4. If the record and file was successfully found update the MediaDBRecord object's metdata attribute with the provided metadata, update the File object and resolve updateMetadataPromise with no arguments.

DomString fileName
The name of the file including path to update metadata for.
Metadata metadata
The metadata to update.
Promise<MediaDBRecord> add ()
Adds a file to the media storage and adds a corresponding MediaDBRecord to the database. If the operation was successful the returned Promise is fullfilled with the stored MediaDBRecord object as argument. A created event is also fired.

This method MUST run the following steps asynchronously:

  1. Create a new Promise, addPromise, return it and run the remaining steps asynchronously.
  2. If the file cannot be added due to an error situation reject addPromise with DOMException TBD and abort the remaining steps.
  3. The file MUST be added to the media storage as a File object according to the following rules:
    • If the fileName parameter is present the name attribute of the new File object added to the media storage is set to the value of the fileName parameter. Note that the name attribute of the new File object includes the path to the file in the file system.
    • If the fileName parameter is not present the name attribute of the File object added to the media storage is generated automatically.
  4. Add a new corresponding MediaDBRecord object to the storage.
  5. fire an event named created at the MediaDB object and resolve addPromise with the stored MediaDBRecord object as argument.

Blob file
The file to add. The file is converted to File object before it is stored. When creating a Blob object, it's mandatory to give it a type, which is a mime type. There are restrictions based on the type so that the mime type must be a valid mime type that corresponds to the MediaDB object, i.e. a valid image, video or audio mime type.
optional DomString fileName
The name of the file to add represented as a string that includes the path. If this parameter is omitted the name of the stored file is automatically generated.
boolean overwrite
States if an existing file with the same name should be overwritten or not. If true overwrite with the new file. If false reject the returned Promise with error TBD. .
Promise<void> delete ()
Deletes a file from the media storage. A deleted event is fired if successful.

This method must return a Promise and then run the following steps asynchronously:

  1. Create a new Promise, deletePromise, return it and run the remaining steps asynchronously.
  2. If the file cannot be deleted due to an error situation reject deletePromise with DOMException TBD and abort the remaining steps.
  3. Delete the file from the media storage and delete the corresponding MediaDBRecord object. fire an event named deleted at the MediaDB object and resolve deletePromise with no arguments.

DomString fileName
The name of the file to delete represented as a string that includes the path.
Promise<unsigned long> count ()
Counts the number of MediaDBRecord objects and passes that number with the resolved Promise.

This method must return a Promise and then run the following steps asynchronously:

  1. Create a new Promise, countPromise, return it and run the remaining steps asynchronously.
  2. If there is an error preventing the method to execute reject countPromise with DOMException TBD and abort the remaining steps.
  3. Count the records in the storage and resolve countPromise with the number of records (unsigned long) as argument.

optional DOMString key
A keypath specification that specifies what field to count. If this argument is defined, it must be 'name', 'size or 'date'. This argument is optional. If omitted all records are counted.

To we need further filtering options?

Promise<unsigned long> freeSpace ()
Gets the amount of free space in the memory available for storing new files, expressed as number of bytes.

This method must return a Promise and then run the following steps asynchronously:

  1. Create a new Promise, freeSpacePromise, return it and run the remaining steps asynchronously.
  2. If there is an error preventing the method to execute reject freeSpacePromise with DOMException TBD and abort the remaining steps.
  3. Calculate the amount of free space usable in the memory and resolve freeSpacePromise with the number of bytes available (unsigned long) as argument.

Scanning

MediaDB automatically scans the file system (internal and/or external) for new and deleted files and updates the media database. The scanning is performed in the background and the scanning frequeny is platform dependent. More capable platforms could immediately discover changes to the files and update the media database, while less capable platforms may scan less frequently. There is also a "manual" scan() method allowing client webapps to start a scanning of the file system in order to assure that the media database is updated.

If the scan finds new files, it reports them to the client webapp with one or more created events. If the scan finds that files have been deleted, it reports them with one or more deleted events.

MediaDB fires a scanstart event when a scan begins and fires a scanend event when the scan is complete. Client webapps can use these events to let the user know that a scan is in progress.

Event handlers

Below is a list of event handler and corresponding event handler event types that MUST be supported as attributes by the MediaDB object.

event handler event handler event type
onready ready
onunavailable unavailable
oncreated created
ondeleted deleted
onscanstart scanstart
onscanend scanend
onpartlyunavailable partlyunavailable

The ready event MUST be fired whenever the MediaDB object becomes ready to use.

The unavailable event MUST be fired whenever the MediaDB object becomes unavailable. The MediaDB state attribute states the detailed reason for the unavailable MediaDB. It is one the values nocard or unmounted.

The created event MUST be fired when one or more files were created. This event MUST implement the CreatedEvent interface.

The deleted event MUST be fired when one or more files were deleted.The deleted event MUST implement the DeletedEvent interface.

When several files are deleted the UA could send one event per deleted file or one event with an array of names of files deleted. This should be further specified but it is a performance issue and depends on platform. Our implementation should be evaluated.

If MediaDB detects that a file has been modified in place (because its size, date or any metadata changed) it treats this as a deletion of the old version and the creation of a new version, and will fire a deleted event followed by a created event.

The created and deleted events are not triggered until the corresponding files have actually been created and deleted and their mediaDBRecord objects have been updated.

The scanstart event MUST be fired when scanning for new and deleted files starts.

The scanend event MUST be fired when scanning for new and deleted files ends.

The partlyunavailable event MUST be fired when a storage area becames permanently unavailable but one or more other storage areas are still available. This happens for example when an SD-card is removed but the internal memory is still available.

The FFOS MediaDB API sends a series of deleted events for all of the files that are no longer available due to the removed storage area. Do we want the same behaviour here? Or do we want another behaviour, e.g. stating that the application is required to call enumerate() to get the fresh staus of the media storage?

Consider to integrate event handling with Service Workers. See https://slightlyoff.github.io/ServiceWorker/spec/service_worker/#extensibility

Interface MediaDBRecord

MediaDB stores a MediaDBRecord object representing each stored media file of the appropriate media type and mime type. Note that the records do not include the file itself, but only its name. Client webapps should use the getFile() method to retrieve the File object.

attribute DOMString name
The filename including file path relative to the media storage root.
attribute DOMString uniqueID
Unique id of the record. This attribute makes it possible to uniquely identify a certain record even if the corresponding media file changes location in the storage.
attribute DOMString type
The file MIME type (extension-based).
attribute unsigned long size
The file size in bytes.
attribute Date? dateAdded
The time the file was added to the media storage. Units are seconds since 1970..
attribute Date? dateModified
The time the file was last modified Units are seconds since 1970.
attribute DOMString displayName
The display name of the file.
attribute DOMString title
The title of the content.
attribute Metadata metadata
Media type specific metadata.
Promise<File> getFile ()
Looks up the file corresponding to this MediaDBRecord and passes it as a File object with the resolved Promise.

This method must return a Promise and then run the following steps asynchronously:

  1. Create a new Promise, getFilePromise, return it and run the remaining steps asynchronously.
  2. Get the specified file from the storage.
  3. If the file can not be found or any other error occurrs reject getFilePromise with DOMException TBD and abort the remaining steps.
  4. If the file was successfully retrieved from the storage let file be a new File object. Resolve getFilePromisePromise with file as argument.

Interface Metadata

Metadata related to the MediaDBRecord objects. This interface is not used directly. Instead Media type specific metadata interfaces that inherits this base metadata interface is defined.

Interface AudioMetadata

Metadata for audio files.

attribute DOMString? album
The album the audio file is from, if any).
attribute unsigned long? albumId
The id of the album the audio file is from, if any.

To be completed with remaining audio metadata attributes for example inspired by Android MediaStore API, audio: http://developer.android.com/reference/android/provider/MediaStore.Audio.AudioColumns.html.

Interface ImageMetadata

Metadata for image files.

attribute unsigned long height
The height of the image in pixels.
attribute unsigned long width
The width of the image in pixels.

To be completed with remaining image metadata attributes for example inspired by Android MediaStore API, image: http://developer.android.com/reference/android/provider/MediaStore.Images.ImageColumns.html

Interface VideoMetadata

Metadata for video files.

attribute DOMString? album
The album the video file is from, if any).
attribute DOMString? artist
The artist who created the video file, if any.

To be completed with remaining video metadata attributes for example inspired by Android MediaStore API, video: http://developer.android.com/reference/android/provider/MediaStore.Video.VideoColumns.html

Interface EnumerateCB

The callback is called once for each entry in the media storage.

MediaDBRecord mediaDBRecord
A record representing a media file.
function cancel
Method allowing client webapps to cancel enumeration. When enumeration has been canceled the Promise returned by the enumerate() method is fulfilled.

Interface CreatedEvent

The CreatedEvent interface represents events related to one or more files of the associated media type added to the file system. The event's attribute is an array of MediaDBRecords corresponding to the added files. When a single file is created (for example when the user takes a picture with the Camera app) this array has only a single element. But when MediaDB scans for new files it may batch multiple records into a single CreatedEvent. If a created event has many records, apps may choose to simply rebuild their UI from scratch with a new call to enumerate() instead of handling the new files one at a time.

readonly attribute sequence<MediaDBRecord> mediaDBRecords
An array of records representing media files added.

Interface DeletedEvent

The DeletedEvent interface represents events related to one or more files of the associated media type deleted from the file system. The event's attribute is an array of the names of the files that have been deleted. As with created events, the array may have a single element or may have many batched elements.

readonly attribute sequence<DomString> name
Array of the names of the files that have been deleted.

Dictionary EnumerateOptions

Options for the enumerate() method. All fields are optional.

EnumerateDirection direction
If this field is omitted, the default is "next", which enumerates entries in ascending order.
DOMString sortKey
A keypath specification that specifies what field to sort on. If this field is defined, it must be 'name', 'size or 'date'. If this filed is omitted, the default is to use the file name as the key.
any filter
Place holder for a filter field.

To we need a filter field? If so, how does it look like? Something like the filter argument in the Tizen Content API, https://developer.tizen.org/documentation/dev-guide/2.2.1?redirect=https://developer.tizen.org/dev-guide/2.2.1/org.tizen.web.appprogramming/html/api_reference/api_reference.htm ?

Enums

MediaType

images
Images media types
videos
Videos media types.
audio
Audio media types.

MediaDBState

ready
MediaDB is available and ready for use by client webapps.
nocard
Unavailable because there is no sd card.
unmounted
Unavailable because the card is unmounted.

EnumerateState

enumerating
Enumeration in progress.
complete
Enumeration completed.
cancelling
Enumeration is cancelling.
cancelled
Enumeration has been cancelled.

EnumerateDirection

next
Enumerates entries in ascending order.
nextunique
TBD
prev
Enumerates entries in descending order.
prevunique
TBD

Acknowledgments

TBD.