Re: Depth image capture workflow proposal for Web

Hi Harald, Ningxin, Giri,
 
> On 19 Mar 2015, at 03:32, Hu, Ningxin <ningxin.hu@intel.com> wrote:
> 
>> From: Harald Alvestrand [mailto:harald@alvestrand.no]
>> Sent: Wednesday, March 18, 2015 11:36 PM
>> 
>> On 03/18/2015 03:27 PM, Kostiainen, Anssi wrote:
>>> Hi Giri, Ningxin, All,
>>> 
>>>> On 18 Mar 2015, at 05:13, Hu, Ningxin <ningxin.hu@intel.com> wrote:
>>> Ningxin - thanks for informing the TF mailing list of the recent
>> developments.
>>> 
>>> [...]
>>> [[
>>> 
>>> [Constructor(MediaStreamTrack track),
>>>  Constructor(MediaStream stream)]
>>> interface ImageCapture {
>>>   // ..
>>> };
>>> 
>>> ]]
>>> 
>>> That'd allow other specs to reuse and extend ImageCapture APIs,
>> 
>> I'd want to ask, though: Does the API that gets reused have to carry
>> everything currently defined on ImageCapture?
>> 
>> In particular, the PhotoCapabilities and PhotoSettings may not be applicable
>> to something that captures something very different from images. (what's
>> the whiteBalanceMode of a depthmap?)
> 
> I think PhotoCapabilities and PhotoSettings are only for color image capture. We may add DepthCapabilities and DepthSettings for depth image capture.

Right. The photoCapabilities getter is currently specific to (color) image capture.

Harald, Giri - what was the reason for not reusing the MediaStreamTrack.getSettings() accessor? To my untrained eye it looks as if that'd allow us to drop the ImageCapture.photoCapabilities attribute that is specific to image capture, and reuse what MediaStreamTrack has to offer (I'm missing earlier discussions around mediacapture-image, so sorry if this has been discussed before).

I mean, instead of:

[[

var track = stream.getVideoTracks()[0];
var ic = new ImageCapture(track);
if (ic.photoCapabilities.redEyeReduction) {
  console.log('long-haul flight mode supported');
  ic.setOptions({ redEyeReductionSetting: true })
    .then(/* ... */); 
}

]]

You'd do:

[[

var track = stream.getVideoTracks()[0];
var ic = new ImageCapture(track);
if (track.getSettings().redEyeReduction) {
  console.log('long-haul flight mode supported');
  ic.setOptions({ redEyeReductionSetting: true })
    .then(/* ... */); 
}

]]

In addition, I believe we'd need to modify the ImageCapture.setOptions() setter if we were to reuse mediacapture-image in the depth image capture workflow. For example:

[[

Promise setOptions ((PhotoSettings or DepthSettings)? settings);

]]

A proposal for the DepthSettings interface is at [2]. The implementation would apply the settings to all matching MediaStreamTracks per kind.

One might chain setOptions() to e.g. first set PhotoSettings followed by DepthSettings if the MediaStream is composed of MediaStreamTracks of different kind, e.g. "video" and "depth":

ic.setOptions({ redEyeReductionSetting: true })
  .then(ic.setOptions({ format: 'rangeInverse' })
  .then(/* ... */));

> This is similar to native camera SDKs. They usually group capture properties to color and depth, and provide property query and set API.

Then it sounds like we should keep the PhotoSettings and DepthSettings property bags separate as proposed above.

>>>  enable extensions such as:
>>> 
>>> [[
>>> 
>>> partial interface FrameGrabEvent {
>>>   readonly attribute DepthMap depthMap; };
>>> 
>>> partial dictionary FrameGrabEventInit {
>>>   DepthMap depthMap;
>>> };
>>> 
>>> ]]
>>> 
>>> I observe the interface name "ImageCapture" may not be a perfect match
>>> for a more generalized API, but the ship may have sailed on the name
>>> (has the feature shipped btw?). If we could still rename things,
>>> "MediaCapture" would perhaps be a better name for a generalised API
>>> that's able to extract also other data than ImageData our of
>>> MediaStreamTracks or MediaStreams. But I don't want to bikeshed about
>>> naming at this time :-)
>> 
>> ImageCapture has not been shipped.

If the feature has not yet shipped, my expectation is there's an opportunity to further improve the API.

> I understand "ImageCapture" name is fine. Supporting depth image capturing or 3D image (color + depth) capturing are all under "Image Capturing".

Sounds reasonable. Personally, I'm fine with the name.

Thanks,

-Anssi

>>> [1] http://w3c.github.io/mediacapture-image/

[2] https://github.com/w3c/mediacapture-depth/issues/72

Received on Thursday, 19 March 2015 08:32:24 UTC