More thoughts on Constraints, and a proposal

I think some of my angst about Constraints is the generality of the
mechanism. That's partly a syntactic problem --- method names are not as
specific as they could be, information is lost as the WebIDL level. It's
also a semantic problem because it means we're forced to pull in
capabilities that particular instances of Constraints don't need or want
(like minimum values for frame sizes in MediaRecorder, async or repeated
constraint application). It limits expressiveness: restricting scalar
parameters to numeric ranges, or strings to a fixed list of values, isn't
the clearest way to express some of the operations we want. (E.g. I think
we still have the situation that setting max width and max height are
treated independently, thus breaking aspect ratio.)

The devolution of constrainable properties to the registry is another
concern. I'm not sure if that was thought necessary to factor out
Constraints into reusable API, or to make the constrainable properties list
more extensible, but it seems better to me to define those properties
inline with the APIs they belong to, or in specs referenced by those APIs.
(It feels like
http://robert.ocallahan.org/2012/05/canvas-getcontext-mistake.html.)

So: instead of having a concrete Constraints interface, what if we made
"Constraints" a set of design guidelines? If we do that, I think we can
make the APIs simpler and fix all the above problems. Of course we would
strive to ensure consistency between consumers for the features they have
in common.

1) Replace Constraints.getCapabilities() with APIs customized for the
particular use-cases where "try it and see" isn't adequate. The generic
Capabilities interface is both generic enough to be hard to use and not
generic enough for edge cases. For example determining from Capabilities
whether "width:500" is supported is error-prone (you need to know which
form of ConstraintValues to check, or check all of them). OTOH you can't
ask MediaRecorder what resolutions it supports when encoding with a
specific format.
2) Replace Constraints.getSettings() with specific individual getters on
the object.
3) Eliminate getConstraints(). The application can determine which
constraints were satisfied by inspecting object state using #2.
4) Eliminate mandatory constraints for the same reason.
5) Replace the "overconstrained" event with events notifying of changes in
specific parts of the state (or a rollup event for all changes).
6) Replace applyConstraints() with the ability to pass one or more
XYZOptions dictionaries to the object constructor, and/or (if we want to
allow ongoing changes to options), an applyOptions() method taking one or
more XYZOptions dictionaries. For async changes it's probably good to keep
the success callback. Without mandatory constraints, there doesn't need to
be an error callback. Resolution of options in multiple dictionaries
follows the existing definition for multiple sets of optional constraints.
7) In the Options dictionaries, give each member its proper type (no
ConstraintValues union). Instead of Range objects, duplicate attributes
with 'min' and 'max' prefixes (simpler, handles one-sided constraints
better (especially when one side never makes sense) and matches other Web
APIs).

Concretely, we could do something like this in MediaRecorder v1:
dictionary MediaRecorderOptions {
  DOMString mimeType; // "optional" constraint
  unsigned long scaleVideoDownToWidth; // "optional" but always honoured in
practice apart from tiny sizes
  unsigned long scaleVideoDownToHeight; // downscaling preserves aspect
ratio if both specified
};
interface MediaRecorder {
  [Constructor] MediaRecorder(MediaStream stream, optional
MediaRecorderOptions options);
  boolean canRecordType(DOMString);
  DOMString mimeType;
  unsigned long getEncodedVideoWidth(MediaStreamTrack); // width of the
last encoded video frame for the track
  unsigned long getEncodedVideoHeight(MediaStreamTrack);
  ...
};
Let's say in MediaRecorder v2 we decide that multiple levels of optional
constraints are needed, and we want to add a feature that reduces FPS, and
we want the browser to report the maximum video size it can encode at for a
given format:
dictionary MediaRecorderOptions {
  ...
  unsigned long reduceToMaxFPS;
};
partial interface MediaRecorder {
  [Constructor] MediaRecorder(MediaStream stream, MediaRecorderOptions
options...);
  unsigned long getCurrentFPS(MediaStreamTrack);
  Size getMaxVideoSize(optional DOMString mimeType); // returns a dictionary
  ...
};

I believe something similar can be done for getUserMedia, but I'm far
enough out along this limb already :-).

Rob
-- 
Jtehsauts  tshaei dS,o n" Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.r"t sS?o  Whhei csha iids  teoa
stiheer :p atroa lsyazye,d  'mYaonu,r  "sGients  uapr,e  tfaokreg iyvoeunr,
'm aotr  atnod  sgaoy ,h o'mGee.t"  uTph eann dt hwea lmka'n?  gBoutt  uIp
waanndt  wyeonut  thoo mken.o w