[mediacapture-image] Add a way to get stream raw pixels

fathyb has just created a new issue for 
https://github.com/w3c/mediacapture-image:

== Add a way to get stream raw pixels ==
`ImageCapture` lacks a way of getting raw camera pixels, I currently 
use `glTexImage2D`+`glReadPixels` but it's not efficient because of 
GPU synchronization and the slow frame roundtrip. I also tried 
`drawImage`+`getImageData` as said in the current draft but it's only 
1ms better on my computer and two times slower on most Android phones.

`ImageBitmap` are great for onscreen stuffs, but they do not provide 
an efficient way to process the camera data on the CPU, for a barcode 
reader for example.
I do not think we should replace it with `ImageData` (again) but maybe
 provide both.

We could have two function, `grabFrame` and `grabImageData`, where 
grabFrame triggers (maybe?) and upload to the GPU and then return an 
`ImageBitmap` and `grabImageData` triggers a CPU read and then return 
an `ImageData`, as most (if not all?) camera drivers store there frame
 in the CPU memory.

The problem with `ImageData` being that if it's allocated each frame 
it could cause GC freezes.

An other solution would be to provide a way (asynchronously?) to get 
an `ImageData` from an `ImageBitmap`, the vendors could then 
efficiently store both if needed and prevent a useless roundtrip.

Please view or discuss this issue at 
https://github.com/w3c/mediacapture-image/issues/147 using your GitHub
 account

Received on Saturday, 21 January 2017 17:06:28 UTC