W3C home > Mailing lists > Public > public-media-capture@w3.org > January 2017

[mediacapture-image] Add a way to get stream raw pixels

From: Fathy Boundjadj via GitHub <sysbot+gh@w3.org>
Date: Sat, 21 Jan 2017 17:06:21 +0000
To: public-media-capture@w3.org
Message-ID: <issues.opened-202326985-1485018380-sysbot+gh@w3.org>
fathyb has just created a new issue for 

== Add a way to get stream raw pixels ==
`ImageCapture` lacks a way of getting raw camera pixels, I currently 
use `glTexImage2D`+`glReadPixels` but it's not efficient because of 
GPU synchronization and the slow frame roundtrip. I also tried 
`drawImage`+`getImageData` as said in the current draft but it's only 
1ms better on my computer and two times slower on most Android phones.

`ImageBitmap` are great for onscreen stuffs, but they do not provide 
an efficient way to process the camera data on the CPU, for a barcode 
reader for example.
I do not think we should replace it with `ImageData` (again) but maybe
 provide both.

We could have two function, `grabFrame` and `grabImageData`, where 
grabFrame triggers (maybe?) and upload to the GPU and then return an 
`ImageBitmap` and `grabImageData` triggers a CPU read and then return 
an `ImageData`, as most (if not all?) camera drivers store there frame
 in the CPU memory.

The problem with `ImageData` being that if it's allocated each frame 
it could cause GC freezes.

An other solution would be to provide a way (asynchronously?) to get 
an `ImageData` from an `ImageBitmap`, the vendors could then 
efficiently store both if needed and prevent a useless roundtrip.

Please view or discuss this issue at 
https://github.com/w3c/mediacapture-image/issues/147 using your GitHub
Received on Saturday, 21 January 2017 17:06:28 UTC

This archive was generated by hypermail 2.4.0 : Friday, 17 January 2020 16:26:39 UTC