Re: MediaStream Recording options

(sorry for the delay; now back from vacation)




On Mon, Jul 22, 2013 at 4:31 AM, Jim Barnett <Jim.Barnett@genesyslab.com>wrote:

> I was thinking of MediaStream constraints, but would want to be consistent
> with Photo in any case, since our use cases are similar. We don't have
> anything like getUserMedia, where the constraints effect which device you
> get.
>


OK. I think we're talking about the same thing -- I read the spec as saying
that MediaStreamConstraints has optional 'video' and 'audio' elements which
can be MediaTrackConstraints objects. I think what we're both referring to
is the dictionary for those MediaTrackConstraint values, correct?
Namely, "A MediaTrackConstraint is a dictionary containing exactly one
key-value pair, where the key must be a valid registered constraint name in
the IANA-hosted RTCWeb Media Constraints registry ..."

Concrete suggestion:

Remove 'AvailableSettings' and make 'RecordingSettings' the common type.
Have it mimic the getUserMedia constraints type. So:

dictionary RecordingSettings {
  DOMString? mimeType;
  MediaTrackConstraint[]? optional;
}

Rename "setOptions()" to "applyConstraints()" to mirror the
MediaStreamTrack API. Remove "getOptions" and introduce a "constraints()"
method returning the current RecordingSettings to mirror the
MediaStreamTrack API.

I think combining the registries and constraint names will overall be
helpful, but there will probably be some names that only make sense in
MediaStreamTrack context, and some that only make sense in the
MediaRecorder context. Should there be a section to try and spell that out?
Having them optional in both places seems to send the right signal -- "the
implementation will just disregard options it doesn't like."


(Aside... speaking of which, I dug around the IANA looking for the
registry, but did not find it. Should it be listed in
http://www.iana.org/protocols? Is there some work to be done to make sure
the names mentioned in getUserMedia are registered there?)



> Jim
>
> On Jul 22, 2013, at 2:31 AM, "Greg Billock" <gbillock@google.com<mailto:
> gbillock@google.com>> wrote:
>
> Thanks, Jim and Harald. Encoding options are a big black hole, obviously.
> :-(
>
> I think if we use MediaTrackConstraints for this, we should mimic the
> MediaStreamTrack API ("constraints()/applyConstraints()") -- or perhaps
> have that mimic the recorder API if getConstraints/setConstraints is
> preferable -- so it's hinted they're the same type. Is that what you had in
> mind when you mentioned hoping to just be able to grab the type and
> semantics from a parallel instance? Or is there another potential spot to
> get it from?
>
>
>
>
>
> On Fri, Jul 19, 2013 at 12:00 PM, Jim Barnett <Jim.Barnett@genesyslab.com
> <mailto:Jim.Barnett@genesyslab.com>> wrote:
> Greg,
>   In general, get/setOptions is the least mature part of the spec.  I’ve
> been hoping that I would be able to inherit the options/settings interface
> from another spec, but so far that hasn’t worked out.   Further comments
> in-line.
>
> From: Greg Billock [mailto:gbillock@google.com<mailto:gbillock@google.com
> >]
> Sent: Friday, July 19, 2013 2:44 PM
> To: public-media-capture@w3.org<mailto:public-media-capture@w3.org>
> Subject: MediaStream Recording options
>
> A couple more questions about the getOptions/setOptions methods.
>
> First, these APIs look dual, but in fact getOptions returns the set of
> available options, not the options that have been set with setOptions.
> Should this be renamed getAvailableOptions instead? That the
> RecordingSettings and AvailableSettings are different types is a bit
> confusing as well. Would renaming the methods be ergonomic enough to go
> with a single RecordingSettings type?
> >> Something like this would make sense, but again I’m hoping to inherit
> the API from another spec, so that all the Recording spec would have to do
> is to define the specific options that it uses.
>
> Having getOptions return the set of current options might still be a good
> idea.
>
> Second. Are the options meant to be user-consumed? That is, is the idea
> that retrieved options can be presented to the user? Or is this meant to
> just be manipulated directly by app code?
> >> They’re mostly intended for app use.  We haven’t really thought about
> presentation to the user.  I wonder if we’d want an additional pretty-print
> name or something like that.
>
> Third. Have you considered options relating to audio sampling? For example
> a "audioBitDepth" and "audioSampleRate" might be really nice. I'm guessing
> this set of options is intended as the spot where further encoding options
> can be added in. How many of these are intended to go here directly as
> settings keys as opposed to, say, params on the mime type?
>
> >> We really haven’t thought about this part.
>
> Fourth. "imageWidth" and "imageHeight" sound like photo capture sizes. The
> comments on AvailableSettings makes it sound like this is capture size for
> video as well, though. Is that right? Will this crop, downsample, or what?
> I think this could use some additional documentation. Should these
> parameters, for video, echo the names in MediaTrackConstraints? "width",
> "height", "frameRate", etc.
> >>  They may have started off as photo settings, and then not been
> properly renamed when photo was moved to a different spec.   Might it
> confuse people if they echo the names for the MediaTrackConstraints?  I’m
> not sure.   There’s been a fair amount of discussion about how the UA
> may/should/must implement MediaStream constraints.  The rough conclusion
> has been that it can either use the settings on the underlying device, or
> do postprocessing, but it’s not allowed to fake data (in other words, it
> can downsample, but not upsample.)  I imagine that similar considerations
> apply here, but we haven’t written it  up yet.  This, again, is something I
> would like to inherit from another spec (i.e. the general syntax and
> semantics of settings.)
>
> Fifth. Is "takePhoto" and associated image parameters redundant to
> MediaStream Image Capture? Any reason to have them in both spots?
> Presumably the pre-encoding image capture will be lower latency and higher
> fidelity.
> >> No, they’re redundant and I have removed them from the editor’s  draft.
>  They were originally in the recording spec, then moved to a separate spec.
>
> Thanks!
>
> -Greg Billock
>
>
>
>
>