RE: Why does media capture require a browser?

So the problem is that using the takePhoto API might have a loop where they grab a frame, process it, and then ask for another frame.  If their processing is fast enough, they might get the same frame again.  With the current API, they'll just have to be careful and use timers to avoid that.  I suppose that we could add a requirement on the UA that consecutive calls to takePhoto must return distinct frames. I don't know how much of a burden on the UA that would be. From what I understood of the discussion, it seems that there are use plenty of cases where the application just wants a single frame, or an occasional frame.  App developers will like the simplicity of the takePhoto API for that.  

If there are use cases where developers want a stream of frames (each frame as it's available, or a frame every n milliseconds, etc.), we could extend the recording API to cover it. It seems  to me that the basic difference between the takePhoto API and the recording API is that the former is a once-and-done request-response API, while the latter is more a more complicated API that returns blocks of data as they're available (or gives you one big Blob when it's done.)  We'll have to be careful to keep the  two APIs distinct, and make it clear what distinguishes them, to avoid confusing people.

- Jim

-----Original Message-----
From: Timothy B. Terriberry [mailto:tterriberry@mozilla.com] 
Sent: Monday, February 25, 2013 10:45 PM
To: public-media-capture@w3.org
Subject: Re: Why does media capture require a browser?

Jim Barnett wrote:
> That makes sense.  If someone has a video processing application that 
> can't afford dropped frames, it cannot be real time.  They will have 
> to call record, save the encoded  file, and postprocess.  For real 
> time applications, they will poll with takeFrame.

I dislike the polling style of APIs because there's an implicit assumption that the processing will _never_ be faster than the rate at which the data is produced. For some video sizes and some processing loads, that's already not true today, and it will become less so in the future (as CPU/GPU power becomes more plentiful, and video framerates stay roughly the same). I think a model like requestAnimationFrame() is far cleaner. It supports throttling when the UA is overloaded (or when a tab is in the background, etc.), but will never trigger more often than actual data is produced.

Received on Tuesday, 26 February 2013 14:32:22 UTC