Fwd: Multimedia User Interfaces: Update

Firefox Developer Team, Greetings.  I would like to describe some new
multimedia user interface features including spatiotemporal content
selection, bookmarking, spatiotemporal zooming, structure-based
navigation, and client-side text-based search into multimedia objects.
 This composition intends to succinctly introduce those ideas and to
describe an illustrative usage scenario, video blogging.

1. Spatiotemporal Content Selection

Spatiotemporal content selection is meant as spatial multimedia
selection, temporal multimedia selection or both.  Spatial multimedia
selection is meant, herein, as indicating rectangular regions on a
video rendering surface.  Temporal multimedia selection is meant as
selecting temporal intervals of multimedia, perhaps on the timeline
upon which the playhead moves.  Users can gesture with keyboard,
mouse, multitouch, voice or NUI to indicate spatial regions and
temporal intervals of multimedia objects.  It occurs that spatial,
temporal and spatiotemporal regions of multimedia content can be
identified by media fragments URI (http://www.w3.org/TR/media-frags).
Extensible context menus are envisioned for spatial, temporal and
spatiotemporal selections.

2. Bookmarking

With regard to bookmarking, users can gesture to place a bookmark, or
point of interest, at a point in a video.  It is envisioned that
bookmarking gestures result in bookmark objects being placed at the
position of the playhead as the user gestures.  After making use of
more of or the entirety of a multimedia object, users can return to
their indicated points of interest, or bookmarks, to, for example,
select temporal intervals around each such point.  Extensible context
menus are envisioned for bookmark points.

3. Spatiotemporal Zooming

With spatiotemporal zooming, users can zoom, from media fragments, to
containing spatial regions and/or temporal intervals or to the
multimedia object that contains the region and/or interval.
Spatiotemporal zooming can make use of tracks that accompany a video,
for example zooming from a search result fragment to a chapter of a
multimedia object that contains the search result media fragment.

4. Structure-Based Navigation

Beyond sequences of chapters are possible outlines or tree-based
structures for multimedia objects.  With such tracks, user interface
implementation ideas include that buttons for chapter traversal can
have menus for indicating the simultaneous traversal options from the
current playhead position.  Spatiotemporal zooming can combine with
structure-based navigation to allow users to zoom from a media
fragment to structural elements of the multimedia object that contain
a media fragment.  For example, a structural model could include
books, parts, chapters, pages, paragraphs and sentences, and, from a
media fragment, a user could zoom to a containing structural element,
and then also navigate by means of those structural elements, based
upon the particular structural model specified in a track.

5. Client-side Text-based Search

By making use of tracks that accompany a multimedia object or of
client-side audio/video indexing and search, client-side text-based
search into documents can include the option of searching into
multimedia objects.  Search results can be indicated by highlighted
portions on the timeline or otherwise visually indicated, perhaps as
per bookmarks.  The finding of text string occurrences in documents
can extend into multimedia objects contained in those documents.

6. Usage Scenario: Video Blogging

Video blogging is an illustrative usage scenario for the above
multimedia user interface features.  A video blogger makes use of a
multimedia search engine for multimedia.  Video fragments are
indicated in the search results.  The user watches a search result
media fragment and decides that they are interested in seeing its
entire video blog article.  The user makes use of zooming to navigate
to a containing section of or to the entire video blog article.  As
the user watches the other video blogger's video, they make use of
bookmarking to place points of interest for later use.  After watching
the video blog, the user makes temporal selections around those
bookmarked points, while perhaps making use of the structural data in
one or more tracks of the video.  The user then makes use of
extensible context menus and utilizes the selected clips in a video
authoring software to compose a video blog article with clips from one
or more multimedia objects.  It also occurs that, by m
 aking use of media fragment URI hyperlinks, users can additionally
tweet about spatiotemporal selections of multimedia.

Other usage scenarios for the new multimedia user interface features
include making use of video from political speeches, news, punditry,
arts and entertainment, civil discourse, and arbitrary multimedia
content, for example when tweeting, blogging or video blogging.



Kind regards,

Adam Sobieski

_______________________________________________
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

From: Adam Sobieski <adamsobieski@hotmail.com>
Date: Mon, Aug 29, 2011 at 7:26 PM
Subject: Multimedia User Interfaces: Update
To: dev-platform@lists.mozilla.org
---------- Forwarded message ----------

Received on Thursday, 1 September 2011 23:23:06 UTC