Re: [depth] depth value encoding proposal

I misunderstood some of the preceding discussion, not realizing that all
mention of depth-to-color mappings had already been removed.  That's good.
 However, the proposed solution (ImageData with special types) has several
remaining problems, as noted in the draft's "Issue 1".

I think depth and color are sufficiently different that it might be better
to create a new DepthData type than to reuse the ImageData type.  A
DepthData object could also indicate whether the units are known (e.g.
millimeters) or unknown, and specify the physical field of view, along with
other valuable metadata.

On Tue, Aug 19, 2014 at 3:31 PM, Harald Alvestrand <>

>  On 08/19/2014 06:01 PM, Benjamin Schwartz wrote:
>  On Tue, Aug 19, 2014 at 10:58 AM, Kostiainen, Anssi <
>> wrote:
>> Hi Benjamin,
>> Thanks for your comments, and sorry for the late reply due to the
>> vacation period.
>> I noticed the following comments (1) and (2) you’ve made, and would like
>> to check the status, and ask your help to fill in the gaps if any:
>> (1) "I do not think this kind of arbitrary mapping is appropriate in a
>> W3C standard.  We should arrange for a natural representation instead, with
>> modifications to Canvas and WebGL if necessary.” [1]
>> To make sure I understood this correctly:
>> Firstly, you're proposing we patch the CanvasRenderingContext2D and make
>> of type ArrayBufferView instead of Uint8ClampedArray to
>> allow the Uint16Array type, correct? Better suggestions?
>  I would suggest going all the way to Float32 as well.
>  Secondly, extend the WebGLRenderingContext along the lines of the
>> LUMINANCE16 extension proposal [2] -- or would you prefer to use the depth
>> component of a GL texture as you previously suggested? Known issues? I’d
>> like to hear your latest thoughts on this and if possible a concrete
>> proposal how you’d prefer this to be spec’d to be practical for developers
>> and logical for implementers.
>  I'd strongly prefer to use a depth component (and also have an option to
> use 32-bit float).  This would raise the GLES version requirement for
> WebGL, but I think this is not an unreasonable requirement for a feature
> that also requires a depth camera!
>> (2) "I don't think getDepthTracks() should return color-video.  If you
>> want to prototype using depth-to-color mappings, the logical way is to
>> treat the depth channel as a distinct color video camera, accessible by the
>> usual device selection mechanisms.” [3]
>> This is what the spec currently says re getDepthTracks():
>> [[
>> The getDepthTracks() method, when invoked, must return a sequence of
>> MediaStreamTrack objects representing the depth tracks in this stream.
>> The getDepthTracks() method must return a sequence that represents a
>> snapshot of all the MediaStreamTrack objects in this stream's track set
>> whose kind is equal to "depth". The conversion from the track set to the
>> sequence is user agent defined and the order does not have to be stable
>> between calls.
>> ]]
>> Do you have a concrete proposal how you’d tighten the prose to clear the
>> confusion?
>  I would not include |getDepthTracks| until we have a depth datatype.
>  Instead, I would add each depth camera as an input device of kind: 'video'
> in the list returned by enumerateDevices(), with its label containing a
> human-readable indication that its color video data is computed from depth
> via a mapping defined by the user agent.
> But once this spec is implemented, we have a depth datatype.... or did I
> misunderstand the proposal?
> I'm afraid that if we spec out a path where depth is treated as a weird
> kind of video, we'll never escape from it - so I'm happy to see such a
> fully-worked proposal that includes both the depth datatype and the depth
> track definition.

Received on Tuesday, 19 August 2014 20:18:00 UTC