RE: MediaStream Recording options

Greg,
I'm going to try to work on standardizing the Settings/Constraints API when people get back from summer vacation.   The rough idea is that each spec would define the settings that make sense for it, so if there are some that make sense for Track but not Recorder, they simply wouldn't be included in the MediaRecorder spec.  (By the way, one of the reasons for splitting image capture into a separate spec is that there are a lot of settings that make sense only for it.)

We will add all the appropriate entries to the IANA registry, but later in the process, when things are more stable.


-          Jim

From: Greg Billock [mailto:gbillock@google.com]
Sent: Monday, July 29, 2013 1:21 PM
To: Jim Barnett
Cc: Harald Alvestrand; public-media-capture@w3.org
Subject: Re: MediaStream Recording options

(sorry for the delay; now back from vacation)



On Mon, Jul 22, 2013 at 4:31 AM, Jim Barnett <Jim.Barnett@genesyslab.com<mailto:Jim.Barnett@genesyslab.com>> wrote:
I was thinking of MediaStream constraints, but would want to be consistent with Photo in any case, since our use cases are similar. We don't have anything like getUserMedia, where the constraints effect which device you get.


OK. I think we're talking about the same thing -- I read the spec as saying that MediaStreamConstraints has optional 'video' and 'audio' elements which can be MediaTrackConstraints objects. I think what we're both referring to is the dictionary for those MediaTrackConstraint values, correct? Namely, "A MediaTrackConstraint is a dictionary containing exactly one key-value pair, where the key must be a valid registered constraint name in the IANA-hosted RTCWeb Media Constraints registry ..."

Concrete suggestion:

Remove 'AvailableSettings' and make 'RecordingSettings' the common type. Have it mimic the getUserMedia constraints type. So:

dictionary RecordingSettings {
  DOMString? mimeType;
  MediaTrackConstraint[]? optional;
}

Rename "setOptions()" to "applyConstraints()" to mirror the MediaStreamTrack API. Remove "getOptions" and introduce a "constraints()" method returning the current RecordingSettings to mirror the MediaStreamTrack API.

I think combining the registries and constraint names will overall be helpful, but there will probably be some names that only make sense in MediaStreamTrack context, and some that only make sense in the MediaRecorder context. Should there be a section to try and spell that out? Having them optional in both places seems to send the right signal -- "the implementation will just disregard options it doesn't like."


(Aside... speaking of which, I dug around the IANA looking for the registry, but did not find it. Should it be listed in http://www.iana.org/protocols? Is there some work to be done to make sure the names mentioned in getUserMedia are registered there?)


Jim

On Jul 22, 2013, at 2:31 AM, "Greg Billock" <gbillock@google.com<mailto:gbillock@google.com><mailto:gbillock@google.com<mailto:gbillock@google.com>>> wrote:

Thanks, Jim and Harald. Encoding options are a big black hole, obviously. :-(

I think if we use MediaTrackConstraints for this, we should mimic the MediaStreamTrack API ("constraints()/applyConstraints()") -- or perhaps have that mimic the recorder API if getConstraints/setConstraints is preferable -- so it's hinted they're the same type. Is that what you had in mind when you mentioned hoping to just be able to grab the type and semantics from a parallel instance? Or is there another potential spot to get it from?




On Fri, Jul 19, 2013 at 12:00 PM, Jim Barnett <Jim.Barnett@genesyslab.com<mailto:Jim.Barnett@genesyslab.com><mailto:Jim.Barnett@genesyslab.com<mailto:Jim.Barnett@genesyslab.com>>> wrote:
Greg,
  In general, get/setOptions is the least mature part of the spec.  I've been hoping that I would be able to inherit the options/settings interface from another spec, but so far that hasn't worked out.   Further comments in-line.
From: Greg Billock [mailto:gbillock@google.com<mailto:gbillock@google.com><mailto:gbillock@google.com<mailto:gbillock@google.com>>]
Sent: Friday, July 19, 2013 2:44 PM
To: public-media-capture@w3.org<mailto:public-media-capture@w3.org><mailto:public-media-capture@w3.org<mailto:public-media-capture@w3.org>>
Subject: MediaStream Recording options

A couple more questions about the getOptions/setOptions methods.

First, these APIs look dual, but in fact getOptions returns the set of available options, not the options that have been set with setOptions. Should this be renamed getAvailableOptions instead? That the RecordingSettings and AvailableSettings are different types is a bit confusing as well. Would renaming the methods be ergonomic enough to go with a single RecordingSettings type?
>> Something like this would make sense, but again I'm hoping to inherit the API from another spec, so that all the Recording spec would have to do is to define the specific options that it uses.

Having getOptions return the set of current options might still be a good idea.

Second. Are the options meant to be user-consumed? That is, is the idea that retrieved options can be presented to the user? Or is this meant to just be manipulated directly by app code?
>> They're mostly intended for app use.  We haven't really thought about presentation to the user.  I wonder if we'd want an additional pretty-print name or something like that.

Third. Have you considered options relating to audio sampling? For example a "audioBitDepth" and "audioSampleRate" might be really nice. I'm guessing this set of options is intended as the spot where further encoding options can be added in. How many of these are intended to go here directly as settings keys as opposed to, say, params on the mime type?

>> We really haven't thought about this part.

Fourth. "imageWidth" and "imageHeight" sound like photo capture sizes. The comments on AvailableSettings makes it sound like this is capture size for video as well, though. Is that right? Will this crop, downsample, or what? I think this could use some additional documentation. Should these parameters, for video, echo the names in MediaTrackConstraints? "width", "height", "frameRate", etc.
>>  They may have started off as photo settings, and then not been properly renamed when photo was moved to a different spec.   Might it confuse people if they echo the names for the MediaTrackConstraints?  I'm not sure.   There's been a fair amount of discussion about how the UA may/should/must implement MediaStream constraints.  The rough conclusion has been that it can either use the settings on the underlying device, or do postprocessing, but it's not allowed to fake data (in other words, it can downsample, but not upsample.)  I imagine that similar considerations apply here, but we haven't written it  up yet.  This, again, is something I would like to inherit from another spec (i.e. the general syntax and semantics of settings.)

Fifth. Is "takePhoto" and associated image parameters redundant to MediaStream Image Capture? Any reason to have them in both spots? Presumably the pre-encoding image capture will be lower latency and higher fidelity.
>> No, they're redundant and I have removed them from the editor's  draft.  They were originally in the recording spec, then moved to a separate spec.

Thanks!

-Greg Billock