Re: Adding "Media Capture Depth Stream Extensions" as TF deliverable

Hi Rob, All,

On 05 Jun 2014, at 15:10, Rob Manson <roBman@buildAR.com> wrote:

> I'm not sure this terminology is clear or correct.
> 
>   A depth stream is an abstraction of a data stream that represents the
>   distance between the camera and the subject for each point of the image.

Agreed, this is too vague. Let’s fix this (below).

> We're not proposing a new type of stream are we? I think we're just planning on using the existing MediaStream interface and extending that by adding a getDepthTracks() method to the interface.

Correct. The proposal is to reuse MediaStream, and add only extensions that do not cause unwanted side effects to the base gUM spec, that is, tap into the extension points defined in the base spec.

> And perhaps we could be clearer about the use of the term "depth track".
> 
>   A depth track is a MediaStreamTrack||object that represents a depth stream

Indeed, this could be clearer too.

> Really we're only proposing that we extend MediaStreamTrack to support "kind" being set to "depth”.

Right.

> So I'd rewrite that text as:
> 
>   This specification extends the getUserMedia based MediaStream interface
>   to enable it to also contain MediaStreamTrack's that have their "kind"
>   attribute set to "depth". These depth based MediaStreamTrack's can be
>   referred to simply as "depth tracks". A depth track is an abstraction of a
>   stream of image frames that can each be converted to ImageData objects
>   which contain an array of pixel data, where each pixel represents the
>   distance between the camera and the objects in the scene for that point in
>   the ImageData array.
> 
> It's a bit more wordy but I think it's more explicit.

I like your proposal, and actually, I think it would make a great Introduction section for the spec.

I reworded it a bit and made the text the Introduction section. I also tried to clarify the “depth stream” and “depth track” in the Terminology section.

I think that now the Introduction + Terminology sections together should be clearer. A pull request with suggested changes is at [1].

Rob, All - please review the PR [1] and send any comments to this list. I’ll incorporate any changes before merging.

> Also, have we defined more info about the "nature and settings of the depth track" that we mention in the constraints section?

This is definitely something that we should clarify, and actually, there’s an open issue for this [2].

Rob - would you be interested in expanding this part of the spec?

> BTW: I just got back from the Augmented World Expo in Santa Clara - the big annual AR event in the US. The unanimous consensus there was that depth sensing was going to be the biggest driving force in the AR industry over the next couple of years. So this is perfect timing for our spec/extension 8)

Sounds good, looking forward to getting more AR experts and other people contribute to this work then :-)

Thanks,

-Anssi

[1] https://github.com/w3c/mediacapture-depth/pull/6/files
[2] https://github.com/w3c/mediacapture-depth/issues/2

Received on Friday, 6 June 2014 08:42:08 UTC