Re: Reusing ConstrainablePattern for mediacapture-image? (was: Depth image capture workflow proposal for Web)

Hi Giri,

> On 25 Mar 2015, at 23:43, Mandyam, Giridhar <mandyam@quicinc.com> wrote:
> 
> I can give my response (once again, colored by my opinions/recollections of discussions that took place during the last 3-4 years):

Thanks for this thorough walkthrough of the historical discussion. Much appreciated.

All - if you have something to add, please chime in.

> The justification for developing a constraints mechanism was provided in http://lists.w3.org/Archives/Public/public-media-capture/2012Feb/0041.html.  I'll also point out early concerns that were discussed with respect to the old 'hints' API surface on gUM:  http://lists.w3.org/Archives/Public/public-media-capture/2012Jan/0014.html.  There was a concern to balance privacy with a means of developers specifying what they would want regarding a capture source.  There may be some valid arguments as to whether the constraints mechanism truly is fingerprinting-proof (or whether it is even worth the effort to try to design API's that are fingerprint-proof), or if developers are really going to be willing/able to take full advantage of the expressivity in constraints.  But I don't want to re-hash those, because we've discussed them at length in the TF before.
> 
> There was also at one time a consideration of bringing in the richer camera settings into MST constraints model:  see https://dvcs.w3.org/hg/dap/raw-file/tip/media-stream-capture/proposals/SettingsAPI_proposal_v6.html.  As I mentioned before, many of the camera settings provided in ImageCapture are not really relevant to the basic WebRTC use cases so it was decided that MST constraints should be confined to a smaller set.  

Ok, so it appears this has been explored in the past in great detail, so we should not rehash this discussion unless someone has new information to add.

Digging into the past discussions with the help of the pointers Giri provided, apparently in circa March 2013 the settings specific to the then-proposed-but-dropped VideoStreamTrack interface [1] (a MediaStreamTrack of kind "video" that can be instantiated) were carved out from the mediacapture-main and folded into the ImageCapture interface [2] that is now defined in the standalone mediacapture-image spec.

In the light of this information it seems the path forward is to extend the mediacapture-image with depth specific settings rather than try to reuse MediaStreamTrack.getSettings().

All - any concerns with this, please let us know.

>> What belongs to the MediaTrackSettings and PhotoSettings may not be clear to a web developer. Also the API shape is different. A potential API ergonomics issue, but not a blocker
> 
> There is an expectation that many developers will not specify constraints beyond {video:true, audio:true} on the gUM call and will be satisfied with the default source settings.  For such developers, any concerns about differing API surface between track.applyContraints and ImageCapture won't come into play.  Maybe a getSetiings() method on the ImageCapture object can be added as opposed to readonly attributes:  as you can tell from Travis's doc from 3 years ago, there has been considerabl changes in the Media Capture and Streams approach to exposing current settings into JS and I haven't really kept up with it.

Adding ImageCapture.getSettings() does sounds good if it does behave similarly to the MediaStreamTrack.getSettings().

Perhaps also s/PhotoSettings/ImageCaptureSettings/ on the same pass since a depth map is not strictly speaking a photo. But I don't feel strongly about this since the dictionary is not visible to a web developer.

Please let me know if you'd like me to create a PR with the proposed changes to be reviewed or whether you'd like to update the spec yourself. Once the proposed changes have landed we'd update the mediacapture-depth to reuse mediacapture-image for [ImageCapture]Settings.

Thanks,

-Anssi 

[1] https://dvcs.w3.org/hg/dap/raw-file/tip/media-stream-capture/proposals/SettingsAPI_proposal_v6.html#idl-def-VideoStreamTrack-1
[2] http://w3c.github.io/mediacapture-image/#idl-def-PhotoCapabilities

Received on Thursday, 26 March 2015 14:30:20 UTC