Re: HTML Media Capture - missing features

Dušan Maliarik wrote:
> Hi,
>
> first of all thank you for the effort, I consider it a move in the right
> direction.
>
> What I miss is an API allowing access to live capture data. The document
> demands user agent to implement a UI and control for the media capture.
> I don't think this is in line with most of other HTML features. What
> makes WWW and HTML different from desktop applications, is that gives an
> opportunity to create any kind of UI, or style of a document in any way
> that HTML and CSS allows. Web developers are free to present interactive
> elements in any form and style they want.
>
> Media Capture API considers this an unnecessary burden thrown at web
> developers, and leaves all this to be implemented inside a browser.
> While this certainly makes things easier for browser developers, it
> takes away many opportunities for innovation on the side of web
> developers. There are many use cases when custom UI, styling and
> scripting is desirable.
>
> In particular, it's missing access to live data from media capture
> device, which could be processed and acted upon, eventually resulting in
> user - browser interaction. Applications ranging from simple media
> filters, augmenting signal on the go, to computer vision and server
> powered speech recognition.

There are two efforts in this area, HTML Media Capture [1] and the 
Streaming Media API [2].

The HTML Media Capture specification builds on the file picker. The 
result is still a complete file, albeit a video or audio file created 
only seconds ago, via the browser's chrome. The purpose of this is to 
allow users to pre-record some media intentionally without requiring a 
web site to handle the recording or any necessary audio/video 
transcoding. For an example, Google have been experimenting with this 
API on Android [3].

On the other hand, the Streaming Media API covers the exact use cases 
you describe above: the ability for a web page to obtain a Stream object 
and assign that object to a <video> or <audio> element as appropriate 
for local playback (and then access the Stream pixel data via the Canvas 
API as required). Opera have been experimenting with this API [4].

Hopefully the links provided are capable of explaining the use cases of 
each proposal and the subtleties and value of having each API. Perhaps 
the HTML Media Capture specification could clearly delineate itself from 
the Streaming Media case since this question has come up before and 
seems to continue to cause a little confusion.

- Rich

[1] http://www.w3.org/TR/media-capture-api/

[2] 
http://www.whatwg.org/specs/web-apps/current-work/multipage/dnd.html#obtaining-local-multimedia-content

[3] 
http://davidbcalhoun.com/2011/android-3-0-honeycomb-is-first-to-implement-the-device-api

[4] http://my.opera.com/core/blog/2011/03/23/webcam-orientation-preview

Received on Monday, 18 April 2011 07:43:40 UTC