Re: use cases not covered by media capture under DAP

Travis Leithead wrote:
> Great points Rich.
>
> Since Canvas is so useful as a post-processing tool in the web today, the benefit that the Media Capture API can provide is high-fidelity images/video to feed to this post-processing tool :-)
>

Right. When it comes to image capture we ideally want access to the 
highest-fidelity resolution of the input data. We might want to:

- use a web app as a bonefide camera that can compete with any native 
camera in which case we want the best quality image data possible.
- use a web app to do some image recognition in which case we get more 
data points to assimilate from a high fidelity image.

It should then be possible to post-process that high fidelity data via 
canvas depending on the intended secondary usage such as:

- thumbnails (scale the capture down to a common width/height and reduce 
the output quality)
- web image formatting (reduce the quality and output the data in a web 
image format such as jpeg or png).
- if it is actually too compute-intensive to do image recognition on the 
high fidelity image data, we might want to scale that down to something 
more performant before applying recognition filters.

The same principles will also apply to audio data - which we will 
handily get via the canvas element also.

We won't be using that high-fidelity data in streaming. The stream will 
degrade to a negotiated bitrate (implicitly determined by the selected 
codec or explicitly declared in the negotiation). For local usage (e.g. 
if I copy a stream to a video element for local playback) then always 
give me the best quality. I can always degrade from there based on my 
actual needs.

> I'll get these added to my scenarios list.

Awesome. Thanks!

>
>> -----Original Message-----
>> From: Rich Tibbett [mailto:richt@opera.com]
>> Sent: Wednesday, November 30, 2011 4:51 PM
>> To: Brian LeRoux
>> Cc: public-media-capture@w3.org
>> Subject: Re: use cases not covered by media capture under DAP
>>
>> Brian LeRoux wrote:
>>> Hey guys, I'm Brian and I work on the PhoneGap project. [1] We
>>> implemented Media Capture. [2] It doesn't cover a number of use cases
>>> so we have a complementary API of our own utility that covers the
>>> common cases for image capturing discovered by PhoneGap app authors.
>>> [3]
>>>
>>> Perhaps these real world cases can be of use.
>>>
>>> In particular things we see all the time:
>>>
>>> - image source (many camera sensors common now, select from photo
>>> lib/gallary VERY common)
>> The HTML Media Capture spec does provide an easy way to obtain 1 or more
>> photo/video/audio files from a file system:
>>
>> http://www.w3.org/TR/html-media-capture/
>>
>> Would this gallery/lib 'capture' need to be real-time for any reason,
>> and therefore manifest itself as a programmatic API for this purpose or
>> would the above suffice?
>>
>>> - image destination (local to app, to the SD card, filesystem, as base64
>> string)
>>
>> 'Download to save' is something the majority of web users are familiar
>> with today. When that mechanism gets triggered (by a user clicking a
>> downloadable file) then files on the web don't have a destination per
>> se. It's very much up to the user at the point of download where they
>> want to save that file.
>>
>> It might be nice to provide a destination hint for a downloadable file
>> but I'm uncertain on the exact use cases for that at this point.
>>
>> Alternatively, we could store all capture data in a common directory on
>> the user's filesystem - so if required the user can go to that common
>> file system location (e.g. ~/Pictures/web/) and obtain their generated
>> image/audio/video data - or a native application can programmatically
>> obtain files from that location since it always knows where to look.
>>
>>> - image quality
>> If you copy the image data to a canvas element you can then get a data
>> URI or blob where you can specify the desired encoding and quality e.g.
>>
>> canvas.toDataURL('image/jpeg', 0.6);
>> // or
>> canvas.toBlob(function(blob) {}, 'image/jpeg', 0.2);
>>
>>> - image rotation
>> If you copy the image data to a canvas element and then obtain its 2D
>> context you can then call rotate() on that context object to rotate the
>> displayed 'image'. You can then obtain the manipulated image back via
>> toDataURL or toBlob as above if you want to generate a file-like object
>> that you can then pass around as required.
>>
>>> - image encoding
>> See image quality above.
>>
>> --
>>
>> I wanted to record the above points in the discussion. I do like the
>> efficacy of canvas to achieve any desired image manipulation (and soon
>> to also be capable of audio manipulation).
>>
>> - Rich
>>
>>
>>> [1] http://phonegap.com
>>> [2]
>> http://docs.phonegap.com/en/1.2.0/phonegap_media_capture_capture.md.html#Ca
>> pture
>>> [3]
>> http://docs.phonegap.com/en/1.2.0/phonegap_camera_camera.md.html#Camera
>>
>
>


-- 
Rich Tibbett
Opera Software ASA

Received on Thursday, 1 December 2011 01:41:03 UTC