RE: Reusing ConstrainablePattern for mediacapture-image? (was: Depth image capture workflow proposal for Web)

I can give my response (once again, colored by my opinions/recollections of discussions that took place during the last 3-4 years):

The justification for developing a constraints mechanism was provided in http://lists.w3.org/Archives/Public/public-media-capture/2012Feb/0041.html.  I'll also point out early concerns that were discussed with respect to the old 'hints' API surface on gUM:  http://lists.w3.org/Archives/Public/public-media-capture/2012Jan/0014.html.  There was a concern to balance privacy with a means of developers specifying what they would want regarding a capture source.  There may be some valid arguments as to whether the constraints mechanism truly is fingerprinting-proof (or whether it is even worth the effort to try to design API's that are fingerprint-proof), or if developers are really going to be willing/able to take full advantage of the expressivity in constraints.  But I don't want to re-hash those, because we've discussed them at length in the TF before.

There was also at one time a consideration of bringing in the richer camera settings into MST constraints model:  see https://dvcs.w3.org/hg/dap/raw-file/tip/media-stream-capture/proposals/SettingsAPI_proposal_v6.html.  As I mentioned before, many of the camera settings provided in ImageCapture are not really relevant to the basic WebRTC use cases so it was decided that MST constraints should be confined to a smaller set.  

> What belongs to the MediaTrackSettings and PhotoSettings may not be clear to a web developer. Also the API shape is different. A potential API ergonomics issue, but not a blocker

There is an expectation that many developers will not specify constraints beyond {video:true, audio:true} on the gUM call and will be satisfied with the default source settings.  For such developers, any concerns about differing API surface between track.applyContraints and ImageCapture won't come into play.  Maybe a getSetiings() method on the ImageCapture object can be added as opposed to readonly attributes:  as you can tell from Travis's doc from 3 years ago, there has been considerabl changes in the Media Capture and Streams approach to exposing current settings into JS and I haven't really kept up with it.

-Giri

-----Original Message-----
From: Kostiainen, Anssi [mailto:anssi.kostiainen@intel.com] 
Sent: Wednesday, March 25, 2015 9:45 AM
To: Mandyam, Giridhar
Cc: Harald Alvestrand; Hu, Ningxin; public-media-capture@w3.org
Subject: Reusing ConstrainablePattern for mediacapture-image? (was: Depth image capture workflow proposal for Web)

Hi Giri, All,

> On 24 Mar 2015, at 01:01, Mandyam, Giridhar <mandyam@quicinc.com> wrote:
> 
> Apologies for late response.  Also, note that some of my answers to the following is based on my recollections of discussions in the group over the past few years and may not be accurate:
> 
>>> what was the reason for not reusing the MediaStreamTrack.getSettings() accessor?
> 
>> getSettings() is specific to the constraints model. Giri was of the opinion that he wanted a simpler model for settings manipulation than the constraints model used on MediaStreamTracks.
> 
> Part of the reason for breaking out image capture functionality into its own spec was so that the MST could have a simpler list of settings that could the types of cameras seen in high-end smartphones and simpler webcams that could be used in typical WebRTC settings.

Thanks for shedding light on the design discussions.

Let's look at a concrete use case I think a web developer might want to tackle to make sure I've understood the current model correctly:

To obtain a blob that represents an image that is at least 640x480 pixels with red eye reduction turned on (omitting some details for brevity):

track.applyConstaints({ width: { min: 640 }, height: { min: 480 } }); var ic = new ImageCapture(track); ic.setOptions({ redEyeReductionSetting: true })
  .then(ic.takePhoto().then(blob => console.log(blob)));

console.log(track.getSetting());
console.log(ic.photoCapabilities);

Is this correct?

Some observations by looking at the above:

What belongs to the MediaTrackSettings and PhotoSettings may not be clear to a web developer. Also the API shape is different. A potential API ergonomics issue, but not a blocker.

I thought a single interface for dealing with settings that provides the accessors, takes a property bag as an argument that can consisting of both generic properties and properties specific to a kind that operate on the MediaStreamTrack level would be easier to work with. The specialized interfaces would inherit from it.

But I'm hearing that did not really work for this case. Pointers to earlier discussions appreciated.

> I can also add that I don't think the constraints paradigm is applicable to camera settings.  Constraints define a sort of language that allows developers to list a range of conditions under which they would like to get access to a capture stream ("Constraints provide a general control surface that allows applications to both select an appropriate source for a track and, once selected, to influence how a source operates.").  In the case of ImageCapture, the source MST has already been created and used for the ImageCapture constructor.  Ergo the constraint model is not applicable.

All - I'm still interested in understanding whether the ContrainablePattern has been implemented. I recall active discussion on the merits of the model, so I'd like to gauge where we stand in terms of implementations now to see what would be the preferred means for depth extensions.

Thanks,

-Anssi

Received on Wednesday, 25 March 2015 21:43:29 UTC