Re: More thoughts on Constraints, and a proposal

+1.

.: Jan-Ivar :.

On 2/3/14 8:25 PM, Robert O'Callahan wrote:
> I think some of my angst about Constraints is the generality of the 
> mechanism. That's partly a syntactic problem --- method names are not 
> as specific as they could be, information is lost as the WebIDL level. 
> It's also a semantic problem because it means we're forced to pull in 
> capabilities that particular instances of Constraints don't need or 
> want (like minimum values for frame sizes in MediaRecorder, async or 
> repeated constraint application). It limits expressiveness: 
> restricting scalar parameters to numeric ranges, or strings to a fixed 
> list of values, isn't the clearest way to express some of the 
> operations we want. (E.g. I think we still have the situation that 
> setting max width and max height are treated independently, thus 
> breaking aspect ratio.)
>
> The devolution of constrainable properties to the registry is another 
> concern. I'm not sure if that was thought necessary to factor out 
> Constraints into reusable API, or to make the constrainable properties 
> list more extensible, but it seems better to me to define those 
> properties inline with the APIs they belong to, or in specs referenced 
> by those APIs. (It feels like 
> http://robert.ocallahan.org/2012/05/canvas-getcontext-mistake.html.)
>
> So: instead of having a concrete Constraints interface, what if we 
> made "Constraints" a set of design guidelines? If we do that, I think 
> we can make the APIs simpler and fix all the above problems. Of course 
> we would strive to ensure consistency between consumers for the 
> features they have in common.
>
> 1) Replace Constraints.getCapabilities() with APIs customized for the 
> particular use-cases where "try it and see" isn't adequate. The 
> generic Capabilities interface is both generic enough to be hard to 
> use and not generic enough for edge cases. For example determining 
> from Capabilities whether "width:500" is supported is error-prone (you 
> need to know which form of ConstraintValues to check, or check all of 
> them). OTOH you can't ask MediaRecorder what resolutions it supports 
> when encoding with a specific format.
> 2) Replace Constraints.getSettings() with specific individual getters 
> on the object.
> 3) Eliminate getConstraints(). The application can determine which 
> constraints were satisfied by inspecting object state using #2.
> 4) Eliminate mandatory constraints for the same reason.
> 5) Replace the "overconstrained" event with events notifying of 
> changes in specific parts of the state (or a rollup event for all 
> changes).
> 6) Replace applyConstraints() with the ability to pass one or more 
> XYZOptions dictionaries to the object constructor, and/or (if we want 
> to allow ongoing changes to options), an applyOptions() method taking 
> one or more XYZOptions dictionaries. For async changes it's probably 
> good to keep the success callback. Without mandatory constraints, 
> there doesn't need to be an error callback. Resolution of options in 
> multiple dictionaries follows the existing definition for multiple 
> sets of optional constraints.
> 7) In the Options dictionaries, give each member its proper type (no 
> ConstraintValues union). Instead of Range objects, duplicate 
> attributes with 'min' and 'max' prefixes (simpler, handles one-sided 
> constraints better (especially when one side never makes sense) and 
> matches other Web APIs).
>
> Concretely, we could do something like this in MediaRecorder v1:
> dictionary MediaRecorderOptions {
>   DOMString mimeType; // "optional" constraint
>   unsigned long scaleVideoDownToWidth; // "optional" but always 
> honoured in practice apart from tiny sizes
>   unsigned long scaleVideoDownToHeight; // downscaling preserves 
> aspect ratio if both specified
> };
> interface MediaRecorder {
>   [Constructor] MediaRecorder(MediaStream stream, optional 
> MediaRecorderOptions options);
>   boolean canRecordType(DOMString);
>   DOMString mimeType;
>   unsigned long getEncodedVideoWidth(MediaStreamTrack); // width of 
> the last encoded video frame for the track
>   unsigned long getEncodedVideoHeight(MediaStreamTrack);
>   ...
> };
> Let's say in MediaRecorder v2 we decide that multiple levels of 
> optional constraints are needed, and we want to add a feature that 
> reduces FPS, and we want the browser to report the maximum video size 
> it can encode at for a given format:
> dictionary MediaRecorderOptions {
>   ...
>   unsigned long reduceToMaxFPS;
> };
> partial interface MediaRecorder {
>   [Constructor] MediaRecorder(MediaStream stream, MediaRecorderOptions 
> options...);
>   unsigned long getCurrentFPS(MediaStreamTrack);
>   Size getMaxVideoSize(optional DOMString mimeType); // returns a 
> dictionary
>   ...
> };
>
> I believe something similar can be done for getUserMedia, but I'm far 
> enough out along this limb already :-).
>
> Rob
> -- 
> Jtehsauts  tshaei dS,o n" Wohfy Mdaon  yhoaus  eanuttehrotraiitny  
> eovni le atrhtohu gthot sf oirng iyvoeu rs ihnesa.r"t sS?o  Whhei csha 
> iids  teoa stiheer :p atroa lsyazye,d  'mYaonu,r  "sGients  uapr,e  
> tfaokreg iyvoeunr, 'm aotr  atnod  sgaoy ,h o'mGee.t" uTph eann dt 
> hwea lmka'n?  gBoutt  uIp  waanndt wyeonut  thoo mken.o w *
> *