[sdw] Breakout discussion: Video search with location (#1130)

rjksmith has just created a new issue for https://github.com/w3c/sdw:

== Breakout discussion: Video search with location ==

Emerging markets in 'mobile video devices', e.g. drones, dashcams, body-worn video and smartphones, are creating a new online community that can share video with metadata content. Displaying metadata, e.g. geolocation, synchronised with video adds valuable context which can significantly improve the viewer's understanding of the associated images. The same information can also be used to create a metadata index of the footage that offers quick and easy access for a search engine user to find relevant content in a video archive, though online search engines do not currently provide this facility.

Would video metadata search be valuable to users and, if so, how best could this be integrated with current services?

**Use Cases**

   1. **Accident Investigation/Motor Insurance**
Automatically identify vehicle collisions in dashcam footage to provide forensic evidence for a police investigation or motor insurance claim.

   1. **Remote Maintenance**
Visually monitor inaccessible assets, e.g. wind turbines, using autonomous drones to create a historical video archive that enables remote expert diagnosis of operational issues.

   1. **Flood Monitoring**
Aggregate video footage from disparate sources to create a historical video archive that allows water levels to be monitored at different locations over time to help predict flooding.

More details of these use cases are given in OGC Ideas issues [#91](https://github.com/opengeospatial/ideas/issues/91) and [#92](https://github.com/opengeospatial/ideas/issues/92).

**Gap Analysis**

The search process can be divided into two parts: acquisition and publication.

   1. **Acquisition**
    A web crawler searches files and updates a search engine database.
       1. **Source file**
        Metadata can be embedded within the media file (in band) or contained separately in a linked file (out of band). For example, [MISB](https://gwg.nga.mil/misb/st_pubs.html) is an in band solution and [WebVMT](https://w3c.github.io/sdw/proposals/geotagging/webvmt/) is out of band - both approaches have pros and cons which should be evaluated.
       1. **Web crawler**
        The web supports three video formats - MPEG, WebM and OGG - and any search solution should address this issue. The source file containing the metadata must be identified and parsed correctly so that relevant details can be added to the search database.

   1. **Publication**
    A search engine responds to a user request.
       1. **User agent**
        A user requests a search from a web page or using a search API, e.g. RESTful interface. The user agent should be able to present the video and metadata content from the search response through a common API in the HTML Document Object Model (DOM). Work is currently underway in the [WICG DataCue](https://github.com/WICG/datacue) activity to investigate and address identified video metadata issues affecting the user agent, including feedback from this discussion.
       1. **Search engine**
        Users should be able to filter content by location and other metadata constraints, and results should be returned with associated location information.


The aim of this discussion is to:
   1. Capture key requirements where user or market needs are not yet met;
   1. Analyse critical gaps that prevent or impair end-to-end metadata search functionality;
   1. Propose solutions with clear explanation;
   1. Agree future activities to investigate outstanding issues and develop working prototypes.

An agenda item has been scheduled to discuss this at the [SDW IG meeting on 25th June 2019](https://www.w3.org/2017/sdwig/meetings/f2f-4.html) in Leuven.

Please view or discuss this issue at https://github.com/w3c/sdw/issues/1130 using your GitHub account

Received on Wednesday, 12 June 2019 12:00:11 UTC