Re: More thoughts on Constraints, and a proposal

Is this a proposal for GUM for Recording or both? I really don’t want to do Recording in a completely different way than we do GUM. 


On Feb 3, 2014, at 6:25 PM, Robert O'Callahan <robert@ocallahan.org> wrote:

> I think some of my angst about Constraints is the generality of the mechanism. That's partly a syntactic problem --- method names are not as specific as they could be, information is lost as the WebIDL level. It's also a semantic problem because it means we're forced to pull in capabilities that particular instances of Constraints don't need or want (like minimum values for frame sizes in MediaRecorder, async or repeated constraint application). It limits expressiveness: restricting scalar parameters to numeric ranges, or strings to a fixed list of values, isn't the clearest way to express some of the operations we want. (E.g. I think we still have the situation that setting max width and max height are treated independently, thus breaking aspect ratio.)
> 
> The devolution of constrainable properties to the registry is another concern. I'm not sure if that was thought necessary to factor out Constraints into reusable API, or to make the constrainable properties list more extensible, but it seems better to me to define those properties inline with the APIs they belong to, or in specs referenced by those APIs. (It feels like http://robert.ocallahan.org/2012/05/canvas-getcontext-mistake.html.)
> 
> So: instead of having a concrete Constraints interface, what if we made "Constraints" a set of design guidelines? If we do that, I think we can make the APIs simpler and fix all the above problems. Of course we would strive to ensure consistency between consumers for the features they have in common.
> 
> 1) Replace Constraints.getCapabilities() with APIs customized for the particular use-cases where "try it and see" isn't adequate. The generic Capabilities interface is both generic enough to be hard to use and not generic enough for edge cases. For example determining from Capabilities whether "width:500" is supported is error-prone (you need to know which form of ConstraintValues to check, or check all of them). OTOH you can't ask MediaRecorder what resolutions it supports when encoding with a specific format.
> 2) Replace Constraints.getSettings() with specific individual getters on the object.
> 3) Eliminate getConstraints(). The application can determine which constraints were satisfied by inspecting object state using #2.
> 4) Eliminate mandatory constraints for the same reason.
> 5) Replace the "overconstrained" event with events notifying of changes in specific parts of the state (or a rollup event for all changes).
> 6) Replace applyConstraints() with the ability to pass one or more XYZOptions dictionaries to the object constructor, and/or (if we want to allow ongoing changes to options), an applyOptions() method taking one or more XYZOptions dictionaries. For async changes it's probably good to keep the success callback. Without mandatory constraints, there doesn't need to be an error callback. Resolution of options in multiple dictionaries follows the existing definition for multiple sets of optional constraints.
> 7) In the Options dictionaries, give each member its proper type (no ConstraintValues union). Instead of Range objects, duplicate attributes with 'min' and 'max' prefixes (simpler, handles one-sided constraints better (especially when one side never makes sense) and matches other Web APIs).
> 
> Concretely, we could do something like this in MediaRecorder v1:
> dictionary MediaRecorderOptions {
>   DOMString mimeType; // "optional" constraint
>   unsigned long scaleVideoDownToWidth; // "optional" but always honoured in practice apart from tiny sizes
>   unsigned long scaleVideoDownToHeight; // downscaling preserves aspect ratio if both specified
> };
> interface MediaRecorder {
>   [Constructor] MediaRecorder(MediaStream stream, optional MediaRecorderOptions options);
>   boolean canRecordType(DOMString);
>   DOMString mimeType;
>   unsigned long getEncodedVideoWidth(MediaStreamTrack); // width of the last encoded video frame for the track
>   unsigned long getEncodedVideoHeight(MediaStreamTrack);
>   ...
> };
> Let's say in MediaRecorder v2 we decide that multiple levels of optional constraints are needed, and we want to add a feature that reduces FPS, and we want the browser to report the maximum video size it can encode at for a given format:
> dictionary MediaRecorderOptions {
>   ...
>   unsigned long reduceToMaxFPS;
> };
> partial interface MediaRecorder {
>   [Constructor] MediaRecorder(MediaStream stream, MediaRecorderOptions options...);
>   unsigned long getCurrentFPS(MediaStreamTrack);
>   Size getMaxVideoSize(optional DOMString mimeType); // returns a dictionary
>   ...
> };
> 
> I believe something similar can be done for getUserMedia, but I'm far enough out along this limb already :-).
> 
> Rob
> -- 
> Jtehsauts  tshaei dS,o n" Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni le atrhtohu gthot sf oirng iyvoeu rs ihnesa.r"t sS?o  Whhei csha iids  teoa stiheer :p atroa lsyazye,d  'mYaonu,r  "sGients  uapr,e  tfaokreg iyvoeunr, 'm aotr  atnod  sgaoy ,h o'mGee.t"  uTph eann dt hwea lmka'n?  gBoutt  uIp  waanndt  wyeonut  thoo mken.o w

Received on Wednesday, 5 February 2014 17:30:01 UTC