Re: use cases not covered by media capture under DAP

On 12/01/2011 02:40 AM, Rich Tibbett wrote:
> Travis Leithead wrote:
>> Great points Rich.
>>
>> Since Canvas is so useful as a post-processing tool in the web today, 
>> the benefit that the Media Capture API can provide is high-fidelity 
>> images/video to feed to this post-processing tool :-)
>>
>
> Right. When it comes to image capture we ideally want access to the 
> highest-fidelity resolution of the input data. We might want to:
>
> - use a web app as a bonefide camera that can compete with any native 
> camera in which case we want the best quality image data possible.
> - use a web app to do some image recognition in which case we get more 
> data points to assimilate from a high fidelity image.
>
> It should then be possible to post-process that high fidelity data via 
> canvas depending on the intended secondary usage such as:
>
> - thumbnails (scale the capture down to a common width/height and 
> reduce the output quality)
> - web image formatting (reduce the quality and output the data in a 
> web image format such as jpeg or png).
> - if it is actually too compute-intensive to do image recognition on 
> the high fidelity image data, we might want to scale that down to 
> something more performant before applying recognition filters.
>
> The same principles will also apply to audio data - which we will 
> handily get via the canvas element also.

You interest me .... where's the current spec for processing audio with 
<canvas>, and what's the opinion of the audio group about that?

>
> We won't be using that high-fidelity data in streaming. The stream 
> will degrade to a negotiated bitrate (implicitly determined by the 
> selected codec or explicitly declared in the negotiation). For local 
> usage (e.g. if I copy a stream to a video element for local playback) 
> then always give me the best quality. I can always degrade from there 
> based on my actual needs.
Note - don't bet on video conferencing being "low quality" always. Among 
the people who want to implement in WebRTC-compatible ways are people 
who think that HD cameras don't have good enough resolution.

At the current state of HW/SW, it's unlikely that we'll see 4K or 8K 
videoconferencing in generic browsers this year, but I would not be 
surprised to see it too eventually arrive.

                     Harald

Received on Monday, 5 December 2011 15:08:20 UTC