Re: Fifth iteration of Settings API now available

Travis,

many thanks for doing this work. There is a lot I like about it, but 
some comments below. Note: the things I don't comment I like - to me 
this seems well on the way to become baked enough for inclusion.

1. Detect new sources
---------------------
In v4, new sources were reported to the application. I think this was 
very valuable - imagine me responding to an invitation immediately, 
using my lap-tops built in (crappy) camera. So while doing the first 
small-talk I manage to find and connect a better camera. Surely it would 
be good if the app became aware of the new camera and could use it? 
Similarly for audio, it usually takes me a minute or two to get my USB 
headset connected, but I would like to start using it once done. Is 
there any way to add this feature (and I am not insisting on it being 
there from day one, just that things are specified so it can be added)?


2. source and remoteSource settings
-----------------------------------
Then I have a philosophical thing I want to discuss a bit. As a 
background: I really like section 4.1 and the discussion there. It 
clarifies a lot of things. But, when you start digging in to the 
settings, it seems that (for video) we end up in:

* Width, Height
* Framrate
* Bitrate
* rotation
* mirror
* zoom
* focusMode
* fillLightMode

You could reason like this in a situation where the video is displayed 
at the other end of a PeerConnection:
* Width and height should be determined by the sink. If the video 
element is 1920*1200, then this should propagate to the source. 
(Likewise if the sink is a recorder). If there are several consumers, 
the dimension of the largest one determines.
* rotation and mirror can be handled at display time (using css transforms).
* zoom, focusMode, fillLightMode and FrameRate are best handled at the 
source - the source side would be in a better position to know if light 
is needed, if the scene requires low or high framerate etc. Note also 
that if these things can be set as remoteSource settings there can be a 
conflict: what if one remoteSource asks for light and another one (from 
another PeerConnection) asks for no light? (I don't know if the same 
reasoning can be applied for mirror/rotation)

What I am getting at is a simplification:
* remove rotation and mirror from the settings - not needed.
* remove width and height from settings (at least from remoteSoource 
settings) and let the consumer(s) control this
* remove zoom, focusMode, fillLightMode and FrameRate from remoteSource 
settings, maintain for localSource

Not that the remote end could still control zoom, focusMode etc. via 
in-app signaling if the app is designed that way - what we remove is the 
need for signaling this using SDP o/a or in RTCP.

This leaves bitrate; but to me bitrate is something that is closely 
related to sending a MediaStreamTrack over a PeerConnection. I would 
like this to be controlled at the sending end of a PeerConnection along 
with things like priority rather than at the receiving end. A reason is 
that you would like the initial SDP offer to contain e.g. BW attributes.

3. Detailed comments
--------------------
a) For the MediaStreamTrack definition, would it not be more natural 
that the "id" was readonly (but that the app could assign a label of its 
own)? And that the "id" must be maintained over a PeerConnection (but 
maybe we don't need to signal label).

b) xxStreamSource/xxStreamRemoteSource/StreamSourceSettings: Should we 
not substitute "Stream" for "Track" in all these cases?

c) VideoStreamTrack has a "facing" attribute. Does this not belong to 
the VideoStreamSource?

d) The VideoStreamSource has attributes like width, height, ... and a 
method "getNumDevices". This does not really make sense to me, the 
different devices could have different widths, heights and so on set. To 
me it seems that getNumDevices (which should work for audio too, 
right?), belongs elsewhere, but I may be getting something wrong

e) We would need to specify if/how the change of one setting alters the 
preference order. IIUC the constraints structure, if used only with 
optional constraints, establishes an order of preference. If the first 
constraint is framerate it says that maintaining framerate within the 
bounds is more important than the following constraints. What happens if 
you later on change one that has lower priority? Does it pop to the top 
of the list?

f) How should it be treated if a device can at setting time fulfill the 
constraint (e.g. framerate) but later, due to e.g. overheating, needs to 
drop below the range? Should a settingserror fire?

g) We could consider removing "enabled" from the definition of the 
MediaStreamTrack. Enabled/selected tracks can also be selected on the 
media element, and having it on two places opens for confusion.

Stefan

PS I think 
http://dvcs.w3.org/hg/dap/raw-file/tip/media-stream-capture/proposals/SettingsAPI_proposal_v5.html 
changed a bit since last time I looked. You could consider naming _v51. 
etc if you do further changes so that referencing is not confused




On 12/01/2012 12:39 AM, Travis Leithead wrote:
> I've completed updates to a Settings API proposal v5.
>
> http://dvcs.w3.org/hg/dap/raw-file/tip/media-stream-capture/proposals/SettingsAPI_proposal_v5.html
>
> The highlights:
> * Simplified settings model
> * Simplified mechanism for discovering other camera/mic devices (before calling getUserMedia)
> * Dropped the device list concept
> * Simpler inheritance for tracks
> * Sources are a property of tracks
> * More complete definition of how settings are expected to work
> * Concepts introduced for constructable tracks and "new" readyState
> * New source objects introduced for tracks belonging to remote media streams
>
> I've already received some feedback that I'll re-post here:
>
> * Some properties for settings may be better suited for the track instead of the source (in v5 all settings are on source objects). These include things like width/height on Video tracks.
> * Some properties of the tracks may be better suited for the source object instead (like "facing").
> * The "get" API might be more useful if it returned the current constraint (or constraint range) applied to the setting (since the properties return the current value).
>
> -Travis
>

Received on Tuesday, 4 December 2012 12:34:16 UTC