- From: Mercurial notifier <cvsmail@w3.org>
- Date: Fri, 30 Nov 2012 23:14:43 +0000
- To: public-dap-commits@w3.org
changeset: 299:8684b783ed17 tag: tip user: travil@travil1.wingroup.windeploy.ntdev.microsoft.com date: Fri Nov 30 15:14:28 2012 -0800 files: media-stream-capture/proposals/SettingsAPI_proposal_v5.html description: [MC&S] Settings v5: Publish final proposal sans re-spec. diff -r 7c56c3056cbc -r 8684b783ed17 media-stream-capture/proposals/SettingsAPI_proposal_v5.html --- a/media-stream-capture/proposals/SettingsAPI_proposal_v5.html Fri Nov 30 15:11:30 2012 -0800 +++ b/media-stream-capture/proposals/SettingsAPI_proposal_v5.html Fri Nov 30 15:14:28 2012 -0800 @@ -0,0 +1,1582 @@ +<!DOCTYPE html> +<html lang="en" dir="ltr"> +<head> + <title>Proposal: Media Capture and Streams Settings API v5</title> + <meta http-equiv="Content-Type" content="text/html;charset=utf-8"> + + + <style>/***************************************************************** + * ReSpec 3 CSS + * Robin Berjon - http://berjon.com/ + *****************************************************************/ + +/* --- INLINES --- */ +em.rfc2119 { + text-transform: lowercase; + font-variant: small-caps; + font-style: normal; + color: #900; +} + +h1 acronym, h2 acronym, h3 acronym, h4 acronym, h5 acronym, h6 acronym, a acronym, +h1 abbr, h2 abbr, h3 abbr, h4 abbr, h5 abbr, h6 abbr, a abbr { + border: none; +} + +dfn { + font-weight: bold; +} + +a.internalDFN { + color: inherit; + border-bottom: 1px solid #99c; + text-decoration: none; +} + +a.externalDFN { + color: inherit; + border-bottom: 1px dotted #ccc; + text-decoration: none; +} + +a.bibref { + text-decoration: none; +} + +cite .bibref { + font-style: normal; +} + +code { + color: #ff4500; +} + + +/* --- --- */ +ol.algorithm { counter-reset:numsection; list-style-type: none; } +ol.algorithm li { margin: 0.5em 0; } +ol.algorithm li:before { font-weight: bold; counter-increment: numsection; content: counters(numsection, ".") ") "; } + +/* --- TOC --- */ +.toc a, .tof a { + text-decoration: none; +} + +a .secno, a .figno { + color: #000; +} + +ul.tof, ol.tof { + list-style: none outside none; +} + +.caption { + margin-top: 0.5em; + font-style: italic; +} + +/* --- TABLE --- */ +table.simple { + border-spacing: 0; + border-collapse: collapse; + border-bottom: 3px solid #005a9c; +} + +.simple th { + background: #005a9c; + color: #fff; + padding: 3px 5px; + text-align: left; +} + +.simple th[scope="row"] { + background: inherit; + color: inherit; + border-top: 1px solid #ddd; +} + +.simple td { + padding: 3px 10px; + border-top: 1px solid #ddd; +} + +.simple tr:nth-child(even) { + background: #f0f6ff; +} + +/* --- DL --- */ +.section dd > p:first-child { + margin-top: 0; +} + +.section dd > p:last-child { + margin-bottom: 0; +} + +.section dd { + margin-bottom: 1em; +} + +.section dl.attrs dd, .section dl.eldef dd { + margin-bottom: 0; +} +</style><style>/* --- ISSUES/NOTES --- */ +div.issue-title, div.note-title { + padding-right: 1em; + min-width: 7.5em; + color: #b9ab2d; +} +div.issue-title { color: #e05252; } +div.note-title { color: #52e052; } +div.issue-title span, div.note-title span { + text-transform: uppercase; +} +div.note, div.issue { + margin-top: 1em; + margin-bottom: 1em; +} +.note > p:first-child, .issue > p:first-child { margin-top: 0 } +.issue, .note { + padding: .5em; + border-left-width: .5em; + border-left-style: solid; +} +div.issue, div.note { + padding: 0.5em; + margin: 1em 0; + position: relative; + clear: both; +} +span.note, span.issue { padding: .1em .5em .15em; } + +.issue { + border-color: #e05252; + background: #fbe9e9; +} +.note { + border-color: #52e052; + background: #e9fbe9; +} + + +</style><style>/* --- WEB IDL --- */ +pre.idl { + border-top: 1px solid #90b8de; + border-bottom: 1px solid #90b8de; + padding: 1em; + line-height: 120%; +} + +pre.idl::before { + content: "WebIDL"; + display: block; + width: 150px; + background: #90b8de; + color: #fff; + font-family: initial; + padding: 3px; + font-weight: bold; + margin: -1em 0 1em -1em; +} + +.idlType { + color: #ff4500; + font-weight: bold; + text-decoration: none; +} + +/*.idlModule*/ +/*.idlModuleID*/ +/*.idlInterface*/ +.idlInterfaceID, .idlDictionaryID, .idlCallbackID, .idlEnumID { + font-weight: bold; + color: #005a9c; +} + +.idlSuperclass { + font-style: italic; + color: #005a9c; +} + +/*.idlAttribute*/ +.idlAttrType, .idlFieldType, .idlMemberType { + color: #005a9c; +} +.idlAttrName, .idlFieldName, .idlMemberName { + color: #ff4500; +} +.idlAttrName a, .idlFieldName a, .idlMemberName a { + color: #ff4500; + border-bottom: 1px dotted #ff4500; + text-decoration: none; +} + +/*.idlMethod*/ +.idlMethType, .idlCallbackType { + color: #005a9c; +} +.idlMethName { + color: #ff4500; +} +.idlMethName a { + color: #ff4500; + border-bottom: 1px dotted #ff4500; + text-decoration: none; +} + +/*.idlParam*/ +.idlParamType { + color: #005a9c; +} +.idlParamName { + font-style: italic; +} + +.extAttr { + color: #666; +} + +/*.idlConst*/ +.idlConstType { + color: #005a9c; +} +.idlConstName { + color: #ff4500; +} +.idlConstName a { + color: #ff4500; + border-bottom: 1px dotted #ff4500; + text-decoration: none; +} + +/*.idlException*/ +.idlExceptionID { + font-weight: bold; + color: #c00; +} + +.idlTypedefID, .idlTypedefType { + color: #005a9c; +} + +.idlRaises, .idlRaises a.idlType, .idlRaises a.idlType code, .excName a, .excName a code { + color: #c00; + font-weight: normal; +} + +.excName a { + font-family: monospace; +} + +.idlRaises a.idlType, .excName a.idlType { + border-bottom: 1px dotted #c00; +} + +.excGetSetTrue, .excGetSetFalse, .prmNullTrue, .prmNullFalse, .prmOptTrue, .prmOptFalse { + width: 45px; + text-align: center; +} +.excGetSetTrue, .prmNullTrue, .prmOptTrue { color: #0c0; } +.excGetSetFalse, .prmNullFalse, .prmOptFalse { color: #c00; } + +.idlImplements a { + font-weight: bold; +} + +dl.attributes, dl.methods, dl.constants, dl.fields, dl.dictionary-members { + margin-left: 2em; +} + +.attributes dt, .methods dt, .constants dt, .fields dt, .dictionary-members dt { + font-weight: normal; +} + +.attributes dt code, .methods dt code, .constants dt code, .fields dt code, .dictionary-members dt code { + font-weight: bold; + color: #000; + font-family: monospace; +} + +.attributes dt code, .fields dt code, .dictionary-members dt code { + background: #ffffd2; +} + +.attributes dt .idlAttrType code, .fields dt .idlFieldType code, .dictionary-members dt .idlMemberType code { + color: #005a9c; + background: transparent; + font-family: inherit; + font-weight: normal; + font-style: italic; +} + +.methods dt code { + background: #d9e6f8; +} + +.constants dt code { + background: #ddffd2; +} + +.attributes dd, .methods dd, .constants dd, .fields dd, .dictionary-members dd { + margin-bottom: 1em; +} + +table.parameters, table.exceptions { + border-spacing: 0; + border-collapse: collapse; + margin: 0.5em 0; + width: 100%; +} +table.parameters { border-bottom: 1px solid #90b8de; } +table.exceptions { border-bottom: 1px solid #deb890; } + +.parameters th, .exceptions th { + color: #fff; + padding: 3px 5px; + text-align: left; + font-family: initial; + font-weight: normal; + text-shadow: #666 1px 1px 0; +} +.parameters th { background: #90b8de; } +.exceptions th { background: #deb890; } + +.parameters td, .exceptions td { + padding: 3px 10px; + border-top: 1px solid #ddd; + vertical-align: top; +} + +.parameters tr:first-child td, .exceptions tr:first-child td { + border-top: none; +} + +.parameters td.prmName, .exceptions td.excName, .exceptions td.excCodeName { + width: 100px; +} + +.parameters td.prmType { + width: 120px; +} + +table.exceptions table { + border-spacing: 0; + border-collapse: collapse; + width: 100%; +} +</style><link href="http://www.w3.org/StyleSheets/TR/W3C-ED" rel="stylesheet"><!--[if lt IE 9]><script src='http://www.w3.org/2008/site/js/html5shiv.js'></script><![endif]--></head> + <body><div class="head"> + <p> + + <a href="http://www.w3.org/"><img width="72" height="48" alt="W3C" src="http://www.w3.org/Icons/w3c_home"></a> + + </p> + <h1 class="title" id="title">Proposal: Media Capture and Streams Settings API v5</h1> + + <h2 id="w3c-editor-s-draft-30-november-2012">Editor's Draft 30 November 2012</h2> + <dl> + <dt>Author:</dt> + <dd><span>Travis Leithead</span>, <a href="http://www.microsoft.com/">Microsoft</a></dd> + </dl> + + <p class="copyright"> + <a href="http://www.w3.org/Consortium/Legal/ipr-notice#Copyright">Copyright</a> © + 2012 + + <a href="http://www.w3.org/"><abbr title="World Wide Web Consortium">W3C</abbr></a><sup>®</sup> + (<a href="http://www.csail.mit.edu/"><abbr title="Massachusetts Institute of Technology">MIT</abbr></a>, + <a href="http://www.ercim.eu/"><abbr title="European Research Consortium for Informatics and Mathematics">ERCIM</abbr></a>, + <a href="http://www.keio.ac.jp/">Keio</a>), All Rights Reserved. + <abbr title="World Wide Web Consortium">W3C</abbr> <a href="http://www.w3.org/Consortium/Legal/ipr-notice#Legal_Disclaimer">liability</a>, + <a href="http://www.w3.org/Consortium/Legal/ipr-notice#W3C_Trademarks">trademark</a> and + <a href="http://www.w3.org/Consortium/Legal/copyright-documents">document use</a> rules apply. + </p> + + + <hr> +</div> + <section class="introductory" id="abstract"><h2>Abstract</h2><p> + This proposal describes additions and suggested changes to the + <a href="http://dev.w3.org/2011/webrtc/editor/getusermedia.html">Media Capture and Streams</a> + specification in order to support the goal of device settings retrieval and modification. This proposal incorporates + feedback from the <abbr title="World Wide Web Consortium">W3C</abbr> TPAC 2012 event and builds on four prior proposals with the same goal + [<a href="http://dvcs.w3.org/hg/dap/raw-file/999605452b3b/media-stream-capture/proposals/SettingsAPI_proposal_v4.html">v4</a>] + [<a href="http://lists.w3.org/Archives/Public/public-media-capture/2012Aug/0143.html">v3</a>] + [<a href="http://lists.w3.org/Archives/Public/public-media-capture/2012Aug/0066.html">v2</a>] + [<a href="http://lists.w3.org/Archives/Public/public-media-capture/2012Jul/0069.html">v1</a>]. + </p></section> + + <section id="toc"><h2 class="introductory">Table of Contents</h2><ul class="toc"><li class="tocline"><a class="tocxref" href="#remove-localmediastream-interface"><span class="secno">1. </span>Remove <code>LocalMediaStream</code> interface</a><ul class="toc"><li class="tocline"><a class="tocxref" href="#rationale"><span class="secno">1.1 </span>Rationale</a></li></ul></li><li class="tocline"><a class="tocxref" href="#media-stream-tracks"><span class="secno">2. </span>Media Stream Tracks</a><ul class="toc"><li class="tocline"><a class="tocxref" href="#updating-mediastreamtrack"><span class="secno">2.1 </span>Updating MediaStreamTrack</a><ul class="toc"><li class="tocline"><a class="tocxref" href="#mediastreamtrack-interface"><span class="secno">2.1.1 </span><code>MediaStreamTrack</code> interface</a></li><li class="tocline"><a class="tocxref" href="#trackreadystateenum-enumeration"><span class="secno">2.1.2 </span>TrackReadyStateEnum enumeration</a></li></ul></li><li class="tocline"><a class="tocxref" href=#creating-derived-tracks"><span class="secno">2.2 </span>Creating Derived Tracks</a><ul class="toc"><li class="tocline"><a class="tocxref" href="#videostreamtrack-interface"><span class="secno">2.2.1 </span><code><span>VideoStreamTrack</span></code> interface</a></li><li class="tocline"><a class="tocxref" href="#audiostreamtrack-interface"><span class="secno">2.2.2 </span><code><span>AudioStreamTrack</span></code> interface</a></li></ul></li></ul></li><li class="tocline"><a class="tocxref" href="#media-stream-sources"><span class="secno">3. </span>Media Stream Sources</a><ul class="toc"><li class="tocline"><a class="tocxref" href="#local-video-and-audio-sources"><span class="secno">3.1 </span>Local Video and Audio Sources</a><ul class="toc"><li class="tocline"><a class="tocxref" href="#videostreamsource-interface"><span class="secno">3.1.1 </span><code><span>VideoStreamSource</span></code> interface</a></li><li class="tocline"><a class="tocxref" href="#videofacingenum-enumeration"><span class="secno">3.1.2 /span><span>VideoFacingEnum</span> enumeration</a></li><li class="tocline"><a class="tocxref" href="#videorotationenum-enumeration"><span class="secno">3.1.3 </span><span>VideoRotationEnum</span> enumeration</a></li><li class="tocline"><a class="tocxref" href="#videomirrorenum-enumeration"><span class="secno">3.1.4 </span><span>VideoMirrorEnum</span> enumeration</a></li><li class="tocline"><a class="tocxref" href="#videofocusmodeenum-enumeration"><span class="secno">3.1.5 </span><span>VideoFocusModeEnum</span> enumeration</a></li><li class="tocline"><a class="tocxref" href="#videofilllightmodeenum-enumeration"><span class="secno">3.1.6 </span><span>VideoFillLightModeEnum</span> enumeration</a></li><li class="tocline"><a class="tocxref" href="#audiostreamsource-interface"><span class="secno">3.1.7 </span><code><span>AudioStreamSource</span></code> interface</a></li></ul></li><li class="tocline"><a class="tocxref" href="#camera-sources-with-high-resolution-picture-modes"><span class="secno">3.2 </span>Camera ources with "high-resolution picture" modes</a><ul class="toc"><li class="tocline"><a class="tocxref" href="#picturestreamsource-interface"><span class="secno">3.2.1 </span><code><span>PictureStreamSource</span></code> interface</a></li><li class="tocline"><a class="tocxref" href="#blobevent-interface"><span class="secno">3.2.2 </span><code>BlobEvent</code> interface</a></li><li class="tocline"><a class="tocxref" href="#blobeventinit-dictionary"><span class="secno">3.2.3 </span>BlobEventInit dictionary</a></li></ul></li><li class="tocline"><a class="tocxref" href="#remote-media-sources"><span class="secno">3.3 </span>Remote Media Sources</a><ul class="toc"><li class="tocline"><a class="tocxref" href="#videostreamremotesource-interface"><span class="secno">3.3.1 </span><code><span>VideoStreamRemoteSource</span></code> interface</a></li><li class="tocline"><a class="tocxref" href="#audiostreamremotesource-interface"><span class="secno">3.3.2 </span><code><span>AudioStreamRemoteSource</span></code> interface</></li></ul></li><li class="tocline"><a class="tocxref" href="#other-settings-out-of-scope-in-this-proposal"><span class="secno">3.4 </span>Other Settings (out-of-scope in this proposal)</a></li></ul></li><li class="tocline"><a class="tocxref" href="#changing-stream-source-settings"><span class="secno">4. </span>Changing Stream Source Settings</a><ul class="toc"><li class="tocline"><a class="tocxref" href="#expectations-around-changing-settings"><span class="secno">4.1 </span>Expectations around changing settings</a></li><li class="tocline"><a class="tocxref" href="#streamsourcesettings-mix-in-interface"><span class="secno">4.2 </span><code>StreamSourceSettings</code> mix-in interface</a><ul class="toc"><li class="tocline"><a class="tocxref" href="#methods"><span class="secno">4.2.1 </span>Methods</a></li><li class="tocline"><a class="tocxref" href="#mediasettingsrange-dictionary"><span class="secno">4.2.2 </span>MediaSettingsRange dictionary</a></li><li class="tocline"><a class="tocxref" href="#mediasettinglist-dictionary"><span class="secno">4.2.3 </span>MediaSettingsList dictionary</a></li></ul></li><li class="tocline"><a class="tocxref" href="#tracking-the-result-of-constraint-application"><span class="secno">4.3 </span>Tracking the result of constraint application</a><ul class="toc"><li class="tocline"><a class="tocxref" href="#mediasettingseventhandlers-mix-in-interface"><span class="secno">4.3.1 </span><code>MediaSettingsEventHandlers</code> mix-in interface</a></li><li class="tocline"><a class="tocxref" href="#mediasettingsevent-interface"><span class="secno">4.3.2 </span><code><span>MediaSettingsEvent</span></code> interface</a></li><li class="tocline"><a class="tocxref" href="#mediasettingseventinit-dictionary"><span class="secno">4.3.3 </span><code><span>MediaSettingsEventInit</span></code> dictionary</a></li></ul></li></ul></li><li class="tocline"><a class="tocxref" href="#constraints-defined-in-this-proposal"><span class="secno">5. </span>Constraints Defined in this Proposal</a><ul class="toc"><liclass="tocline"><a class="tocxref" href="#video-constraints"><span class="secno">5.1 </span>Video Constraints</a><ul class="toc"><li class="tocline"><a class="tocxref" href="#videoconstraints-dictionary"><span class="secno">5.1.1 </span>VideoConstraints dictionary</a></li></ul></li><li class="tocline"><a class="tocxref" href="#audio-constraints"><span class="secno">5.2 </span>Audio Constraints</a><ul class="toc"><li class="tocline"><a class="tocxref" href="#audioconstraints-dictionary"><span class="secno">5.2.1 </span>AudioConstraints dictionary</a></li></ul></li><li class="tocline"><a class="tocxref" href="#common-sub-constraint-structures"><span class="secno">5.3 </span>Common sub-constraint structures</a><ul class="toc"><li class="tocline"><a class="tocxref" href="#minmaxulongsubconstraint-dictionary"><span class="secno">5.3.1 </span><code><span>MinMaxULongSubConstraint</span></code> dictionary</a></li><li class="tocline"><a class="tocxref" href="#minmaxfloatsubconstraint-dictionary"><span class="secno">.3.2 </span><code><span>MinMaxFloatSubConstraint</span></code> dictionary</a></li></ul></li></ul></li><li class="tocline"><a class="tocxref" href="#example-usage-scenarios"><span class="secno">6. </span>Example usage scenarios</a><ul class="toc"><li class="tocline"><a class="tocxref" href="#getting-access-to-a-video-and-or-audio-device-if-available"><span class="secno">6.1 </span>Getting access to a video and/or audio device (if available)</a></li><li class="tocline"><a class="tocxref" href="#previewing-the-local-video-audio-in-html5-video-tag----scenario-is-unchanged"><span class="secno">6.2 </span>Previewing the local video/audio in HTML5 video tag -- scenario is unchanged</a></li><li class="tocline"><a class="tocxref" href="#applying-resolution-constraints"><span class="secno">6.3 </span>Applying resolution constraints</a></li><li class="tocline"><a class="tocxref" href="#changing-zoom-in-response-to-user-input"><span class="secno">6.4 </span>Changing zoom in response to user input:</a></li><li class="toline"><a class="tocxref" href="#adding-the-local-media-tracks-into-a-new-media-stream"><span class="secno">6.5 </span>Adding the local media tracks into a new media stream:</a></li><li class="tocline"><a class="tocxref" href="#take-a-picture-show-the-picture-in-an-image-tag"><span class="secno">6.6 </span>Take a picture, show the picture in an image tag:</a></li><li class="tocline"><a class="tocxref" href="#show-a-newly-available-device"><span class="secno">6.7 </span>Show a newly available device</a></li><li class="tocline"><a class="tocxref" href="#show-all-available-video-devices"><span class="secno">6.8 </span>Show all available video devices:</a></li></ul></li><li class="tocline"><a class="tocxref" href="#changes"><span class="secno">7. </span>Changes</a></li><li class="tocline"><a class="tocxref" href="#acknowledgements"><span class="secno">8. </span>Acknowledgements</a></li></ul></section> + + <section id="remove-localmediastream-interface"> + <!--OddPage--><h2><span class="secno">1. </span>Remove <code>LocalMediaStream</code> interface</h2> + <p>In this proposal, the derived LocalMediaStream interface is removed. Rather than returning a LocalMediaStream + instance in the NavigatorUserMediaSuccessCallback, a vanilla MediaStream object is returned. The primary difference + is in the tracks contained in that MediaStream object. + </p> + + <section id="rationale"> + <h3><span class="secno">1.1 </span>Rationale</h3> + + The LocalMediaStream object currently extends MediaStream by adding a single method "stop()". In my prior proposals, this + object was radically altered in order to facilitate several goals: + <dl> + <dt>Provide a predictable home for developers to find and modify device settings</dt> + <dd>A previous proposal went out of its way to strongly associate LocalMediaStream objects with devices. This + seemed like a good design because local device configuration is always on the local media stream. This made + for a stable, dependable API surface for all local media stream instances (no guesswork). + </dd> + <dt>Prevent track-list mutations</dt> + <dd>A previous proposal also removed the track lists on local media streams (resulting in some dramatic inheritance + changes). Mutable tracks lists on LocalMediaStream objects seemed like the wrong design considering the current + thinking that a getUserMedia request would only ever produce a LocalMediaStream with at most one audio or video + track. + </dd> + </dl> + + <p>Some feedback even suggested re-considering the "at most one video/audio track per request to getUserMedia".</p> + + <p>While thinking about these goals and the feedback, I began to consider a few things:</p> + + <dl> + <dt>Device-centric tracks</dt> + <dd>With tracks supplemented with device-characteristics (duck-typing), the LocalMediaStream's stop() API was a + convenience feature for stopping all tracks backed by a device on the LocalMediaStream object. With device- + centric tracks a stop() API should be present on the tracks themselves. + </dd> + <dt>Mutable track lists</dt> + <dd>Mutable track lists were not a desirable feature while I was locked into considering the LocalMediaStream + as strongly associated with device-control. What is actually necessary, is that there is a something immutable + associated with devices--that "something" doesn't necessarily need to be a LocalMediaStream or any MediaStream-like + object at all! Once I unlocked that line of thinking, I began to experiment with the notion of a device list + which then ultimately brought back a use-case for having mutable track lists for MediaStream objects. (It did not + bring back a need for LocalMediaStream objects themselves though.) + </dd> + <dt>Work flow for access to additional device streams</dt> + <dd>It is now understood that to request additional streams for different devices (e.g., the second camera on a + dual-camera mobile phone), one must invoke getUserMedia a second time. In my prior proposal, this would result + in a separate LocalMediaStream instance. At this point there are two LocalMediaStream objects each with their + own devices. While this was nice for consistency of process, it was a challenge in terms of use of the objects + with a MediaStream consumer like the Video tag. + + <p>To illustrate this challenge, consider how my prior proposal required a re-hookup of the MediaStream + to a video tag consumer:</p> + + <ol> + <li>First request to getUserMedia</li> + <li>LocalMediaStream (1) obtained from success callback</li> + <li>createObjectURL and preview in a video tag</li> + <li>Second call to getUserMedia</li> + <li>LocalMediaStream (2) obtained from success callback</li> + <li>createObjectURL and preview in same video tag</li> + </ol> + + <p>Note that this process has to bind a completely new LocalMediaStream to the video tag a second time (if + re-using the same video tag) only because the second LocalMediaStream object was different than the + first.</p> + + <p>It is much more efficient for developer code to simply add/remove tracks to a MediaStream that are + relevant, without needing to change the consumer of the MediaStream.</p> + </dd> + <dt>Usage of getUserMedia for permission rather than for additional device access</dt> + <dd>The getUserMedia method is the gateway for permission to media. This proposal does not suggest + changing that concept. It <em>does</em> suggest, however, that more information can be made available for + discovery of additional devices within the approved "category" or "type" of media, and provide a way to + obtain those additional devices without needing to go through the "permissions" route (i.e., getUserMedia). + </dd> + <dt>Importance of restricting control to LocalMediaStream</dt> + <dd>Upon reflection of the feedback around the prior proposal, the relative importance of restricting control + of the devices associated with tracks on the LocalMediaStream to <em>only</em> the LocalMediaStream did not + seem as vital, insofar as the device-level access via the track is not directly available through a + PeerConnection to a remote browser. + </dd> + </dl> + </section> + </section> + + <section id="media-stream-tracks"> + <!--OddPage--><h2><span class="secno">2. </span>Media Stream Tracks</h2> + + <p>With changes to <code>getUserMedia</code> to support a synchronous API, this proposal enables developer code to + directly create Media Stream Tracks. It also introduces the concept of the <code>"new"</code> readyState for tracks, + a state which signals that the specified track is not connected to a source. + </p> + + <p>All tracks now have a <code>source</code> attribute, which is used to access a source object. The source object can be + used to read additional settings about the source (content) of a track and to alter the source (content) of the track. + This proposal describes local media device sources (cameras and microphones), and a skeleton description for a + remote media device source (tracks that originate from a peer connection). Media device source objects are described + in the next section. + </p> + + <p>Below is the new track hierarchy. It is somewhat simplified due to the exclusion of source objects: + </p> + + <ul> + <li>MediaStreamTrack + <ul> + <li>VideoStreamTrack</li> + <li>AudioStreamTrack</li> + </ul> + </li> + </ul> + + <section id="updating-mediastreamtrack"> + <h3><span class="secno">2.1 </span>Updating MediaStreamTrack</h3> + + <p>This section defines <dfn id="dfn-mediastreamtrack">MediaStreamTrack</dfn> in order to add the new <code>"new"</code> state and associated + event handlers. The definition is otherwise + identical to the current definition except that the defined constants are replaced by strings (using an enumerated type). + </p> + + <section id="mediastreamtrack-interface"> + <h4><span class="secno">2.1.1 </span><code>MediaStreamTrack</code> interface</h4> + <pre class="idl"><span class="idlInterface" id="idl-def-MediaStreamTrack">interface <span class="idlInterfaceID">MediaStreamTrack</span> : <span class="idlSuperclass"><a>EventTarget</a></span> { +<span class="idlAttribute"> attribute <span class="idlAttrType"><a>DOMString</a></span> <span class="idlAttrName"><a href="#widl-MediaStreamTrack-id">id</a></span>;</span> +<span class="idlAttribute"> readonly attribute <span class="idlAttrType"><a>DOMString</a></span> <span class="idlAttrName"><a href="#widl-MediaStreamTrack-kind">kind</a></span>;</span> +<span class="idlAttribute"> readonly attribute <span class="idlAttrType"><a>DOMString</a></span> <span class="idlAttrName"><a href="#widl-MediaStreamTrack-label">label</a></span>;</span> +<span class="idlAttribute"> attribute <span class="idlAttrType"><a>boolean</a></span> <span class="idlAttrName"><a href="#widl-MediaStreamTrack-enabled">enabled</a></span>;</span> +<span class="idlAttribute"> readonly attribute <span class="idlAttrType"><a class="idlType" href="#idl-def-TrackReadyStateEnum"><code>TrackReadyStateEnum</code></a></span> <span class="idlAttrName"><a href="#widl-MediaStreamTrack-readyState">readyState</a></span>;</span> +<span class="idlAttribute"> attribute <span class="idlAttrType"><a>EventHandler</a></span> <span class="idlAttrName"><a href="#widl-MediaStreamTrack-onstart">onstart</a></span>;</span> +<span class="idlAttribute"> attribute <span class="idlAttrType"><a>EventHandler</a></span> <span class="idlAttrName"><a href="#widl-MediaStreamTrack-onmute">onmute</a></span>;</span> +<span class="idlAttribute"> attribute <span class="idlAttrType"><a>EventHandler</a></span> <span class="idlAttrName"><a href="#widl-MediaStreamTrack-onunmute">onunmute</a></span>;</span> +<span class="idlAttribute"> attribute <span class="idlAttrType"><a>EventHandler</a></span> <span class="idlAttrName"><a href="#widl-MediaStreamTrack-onended">onended</a></span>;</span> +};</span></pre><section><h5 id="attributes">Attributes</h5><dl class="attributes"><dt id="widl-MediaStreamTrack-id"><code>id</code> of type <span class="idlAttrType"><a>DOMString</a></span></dt><dd>Provides a mechanism for developers to identify this track and to reference it by <code>getTrackById</code>. (This is a preliminary definition, but is + expected in the latest editor's draft soon.) + </dd><dt id="widl-MediaStreamTrack-kind"><code>kind</code> of type <span class="idlAttrType"><a>DOMString</a></span>, readonly</dt><dd>See <a href="http://dev.w3.org/2011/webrtc/editor/getusermedia.html#widl-MediaStreamTrack-kind">kind</a> definition in the current editor's draft.</dd><dt id="widl-MediaStreamTrack-label"><code>label</code> of type <span class="idlAttrType"><a>DOMString</a></span>, readonly</dt><dd>See <a href="http://dev.w3.org/2011/webrtc/editor/getusermedia.html#widl-MediaStreamTrack-label">label</a> definition in the current editor's draft.</dd><dt id="widl-MediaStreamTrack-enabled"><code>enabled</code> of type <span class="idlAttrType"><a>boolean</a></span></dt><dd>See <a href="http://dev.w3.org/2011/webrtc/editor/getusermedia.html#widl-MediaStreamTrack-enabled">enabled</a> definition in the current editor's draft.</dd><dt id="widl-MediaStreamTrack-readyState"><code>readyState</code> of type <span class="idlAttrType"><a class="idlType" href="#idl-def-TrackReadyStatenum"><code>TrackReadyStateEnum</code></a></span>, readonly</dt><dd>The track's current state. Tracks start off in the <code>"new"</code> state after being instantiated. + <p>State transitions are as follows:</p> + <ul> + <li><strong>new -> live</strong> The user has approved access to this track and a media device source is now attached and streaming data.</li> + <li><strong>new -> ended</strong> The user rejected this track (did not approve its use).</li> + <li><strong>live -> muted</strong> The source is temporarily suspended (cannot provide streaming data).</li> + <li><strong>live -> ended</strong> The stream has ended (for various reasons).</li> + <li><strong>muted -> live</strong> The stream has resumed.</li> + <li><strong>muted -> ended</strong> The stream was suspended and will no longer be able to provide any further data.</li> + </ul> + </dd><dt id="widl-MediaStreamTrack-onstart"><code>onstart</code> of type <span class="idlAttrType"><a>EventHandler</a></span></dt><dd>Event handler for the <code>start</code> event. The <code>start</code> event is fired when this track transitions + from the <code>"new"</code> state to the <code>"live"</code> state. + <div class="issue"><div class="issue-title"><span>Issue 1</span></div><p><strong>Issue: </strong> When working with multiple <code>"new"</code> tracks, I found that I wanted to have a more centralized + place to be notified when getUserMedia would activate all the tracks in a media stream. Perhaps there's a convenience handler + somewhere else, for example on the MediaStream? There's some work flows to consider here before landing a final design... + </p></div> + </dd><dt id="widl-MediaStreamTrack-onmute"><code>onmute</code> of type <span class="idlAttrType"><a>EventHandler</a></span></dt><dd>See <a href="http://dev.w3.org/2011/webrtc/editor/getusermedia.html#widl-MediaStreamTrack-onmute">onmute</a> definition in the current editor's draft. + </dd><dt id="widl-MediaStreamTrack-onunmute"><code>onunmute</code> of type <span class="idlAttrType"><a>EventHandler</a></span></dt><dd>See <a href="http://dev.w3.org/2011/webrtc/editor/getusermedia.html#widl-MediaStreamTrack-onunmute">onunmute</a> definition in the current editor's draft. + </dd><dt id="widl-MediaStreamTrack-onended"><code>onended</code> of type <span class="idlAttrType"><a>EventHandler</a></span></dt><dd>See <a href="http://dev.w3.org/2011/webrtc/editor/getusermedia.html#widl-MediaStreamTrack-onended">onended</a> definition in the current editor's draft. + </dd></dl></section> + </section> + + <section id="trackreadystateenum-enumeration"> + <h4><span class="secno">2.1.2 </span>TrackReadyStateEnum enumeration</h4> + <pre class="idl"><span class="idlEnum" id="idl-def-TrackReadyStateEnum">enum <span class="idlEnumID">TrackReadyStateEnum</span> { + "<span class="idlEnumItem">new</span>", + "<span class="idlEnumItem">live</span>", + "<span class="idlEnumItem">muted</span>", + "<span class="idlEnumItem">ended</span>" +};</span></pre><table class="simple"><tr><th colspan="2">Enumeration description</th></tr><tr><td><code>new</code></td><td>The track type is new and has not been initialized (connected to a source of any kind). This state implies that + the track's label will be the empty string.</td></tr><tr><td><code>live</code></td><td>See the definition of the <a href="">LIVE</a> constant in the current editor's draft.</td></tr><tr><td><code>muted</code></td><td>See the definition of the <a href="">MUTED</a> constant in the current editor's draft.</td></tr><tr><td><code>ended</code></td><td>See the definition of the <a href="">ENDED</a> constant in the current editor's draft.</td></tr></table> + </section> + </section> + + <section id="creating-derived-tracks"> + <h3><span class="secno">2.2 </span>Creating Derived Tracks</h3> + + <p><a class="internalDFN" href="#dfn-mediastreamtrack">MediaStreamTrack</a> objects cannot be instantiated directly. To create an instance of a <a class="internalDFN" href="#dfn-mediastreamtrack">MediaStreamTrack</a>, one of + its derived track types may be instantiated directly. These derived types are defined in this section. Each of these + track types has general IDL attributes specific to all tracks of the given type as well as a mechanism to obtain the + device object that is providing the source for this track. + </p> + + <div class="note"><div class="note-title"><span>Note</span></div><p><strong>Note: </strong> I'm intentionally keeping these interfaces as sparse as possible. Features of the video/audio tracks that + are settings (generally mutable) have been moved to the track's device source instead. + </p></div> + + <p>It's important to note that the camera's <q>green light</q> doesn't come on when a new track is created; nor does the user get + prompted to enable the camera/microphone. Those actions only happen after the developer has requested that a media stream containing + <code>"new"</code> tracks be bound to a source via <code>getUserMedia</code>. Until that point tracks are inert. + </p> + + <section id="videostreamtrack-interface"> + <h4><span class="secno">2.2.1 </span><code><dfn id="dfn-videostreamtrack">VideoStreamTrack</dfn></code> interface</h4> + + <p>VideoStreamTrack objects are of <code>kind</code> "video".</p> + + <div class="note"><div class="note-title"><span>Note</span></div><p><strong>Example: </strong><a class="internalDFN" href="#dfn-videostreamtrack">VideoStreamTrack</a> objects are instantiated in JavaScript using the new operator: <br> + <span style="white-space: pre;"><code>new VideoStreamTrack();</code></span> + </p></div> + + <div class="issue"><div class="issue-title"><span>Issue 2</span></div><p><strong>Issue: </strong> It's been suggested that these track constructors can take optional constraints. Such constraints + could be stored until activated by getUserMedia to help the user select what characteristics to use when pre-configuring a device source + and attaching it to the track. These are not added in this proposal, but could be added later-on. + </p></div> + + <pre class="idl"><span class="idlInterface" id="idl-def-VideoStreamTrack">[<span class="extAttr">Constructor</span>] +interface <span class="idlInterfaceID">VideoStreamTrack</span> : <span class="idlSuperclass"><a class="idlType" href="#idl-def-MediaStreamTrack"><code>MediaStreamTrack</code></a></span> { +<span class="idlAttribute"> readonly attribute <span class="idlAttrType"><a class="idlType" href="#idl-def-VideoFacingEnum"><code>VideoFacingEnum</code></a></span> <span class="idlAttrName"><a href="#widl-VideoStreamTrack-facing">facing</a></span>;</span> +<span class="idlAttribute"> readonly attribute <span class="idlAttrType"><a class="idlType" href="#idl-def-VideoStreamSource"><code>VideoStreamSource</code></a>?</span> <span class="idlAttrName"><a href="#widl-VideoStreamTrack-source">source</a></span>;</span> +};</span></pre><section><h5 id="attributes-1">Attributes</h5><dl class="attributes"><dt id="widl-VideoStreamTrack-facing"><code>facing</code> of type <span class="idlAttrType"><a class="idlType" href="#idl-def-VideoFacingEnum"><code>VideoFacingEnum</code></a></span>, readonly</dt><dd>From the user's perspective, this attribute describes whether this camera is pointed toward the + user ("user") or away from the user ("environment"). If this information cannot be reliably obtained, + for example from a USB external camera, or if the VideoStreamTrack's <code>readyState</code> is <code>"new"</code>, + the value <code>"unknown"</code> is returned. + </dd><dt id="widl-VideoStreamTrack-source"><code>source</code> of type <span class="idlAttrType"><a class="idlType" href="#idl-def-VideoStreamSource"><code>VideoStreamSource</code></a></span>, readonly, nullable</dt><dd>Returns the <a class="internalDFN" href="#dfn-videostreamsource">VideoStreamSource</a> object providing the source for this track (if available). A <a class="internalDFN" href="#dfn-videostreamsource">VideoStreamSource</a> may be + a camera, a peer connection, or a local image or video file. Some <a class="internalDFN" href="#dfn-videostreamtrack">VideoStreamTrack</a> sources may not expose a + <a class="internalDFN" href="#dfn-videostreamsource">VideoStreamSource</a> object, in which case this property must return <code>null</code>. When a <a class="internalDFN" href="#dfn-videostreamtrack">VideoStreamTrack</a> is first + created, and while it remains in the <code>"new"</code> state, the <code>source</code> attribute must return <code>null</code>. + </dd></dl></section> + </section> + + <section id="audiostreamtrack-interface"> + <h4><span class="secno">2.2.2 </span><code><dfn id="dfn-audiostreamtrack">AudioStreamTrack</dfn></code> interface</h4> + + <p>AudioStreamTrack objects are of <code>kind</code> "audio".</p> + + <div class="note"><div class="note-title"><span>Note</span></div><p><strong>Example: </strong><a class="internalDFN" href="#dfn-audiostreamtrack">AudioStreamTrack</a> objects are instantiated in JavaScript using the new operator: <br> + <span style="white-space: pre;"><code>new AudioStreamTrack();</code></span> + </p></div> + + <pre class="idl"><span class="idlInterface" id="idl-def-AudioStreamTrack">[<span class="extAttr">Constructor</span>] +interface <span class="idlInterfaceID">AudioStreamTrack</span> : <span class="idlSuperclass"><a class="idlType" href="#idl-def-MediaStreamTrack"><code>MediaStreamTrack</code></a></span> { +<span class="idlAttribute"> readonly attribute <span class="idlAttrType"><a>unsigned long</a></span> <span class="idlAttrName"><a href="#widl-AudioStreamTrack-level">level</a></span>;</span> +<span class="idlAttribute"> readonly attribute <span class="idlAttrType"><a class="idlType" href="#idl-def-AudioStreamSource"><code>AudioStreamSource</code></a>?</span> <span class="idlAttrName"><a href="#widl-AudioStreamTrack-source">source</a></span>;</span> +};</span></pre><section><h5 id="attributes-2">Attributes</h5><dl class="attributes"><dt id="widl-AudioStreamTrack-level"><code>level</code> of type <span class="idlAttrType"><a>unsigned long</a></span>, readonly</dt><dd>The current level of audio that the microphone is picking up at this moment (if this is an AudioDeviceTrack), + or the current level of audio flowing through the track (generally) otherwise. Will return 0 if this track is + in the <code>"new"</code> state. The relative strength (amplitude) of the level is proportional to the <code>gain</code> of the + audio source device (e.g., to increase the pick-up of the microphone, increase the gain setting). + </dd><dt id="widl-AudioStreamTrack-source"><code>source</code> of type <span class="idlAttrType"><a class="idlType" href="#idl-def-AudioStreamSource"><code>AudioStreamSource</code></a></span>, readonly, nullable</dt><dd>Returns the <a class="internalDFN" href="#dfn-audiostreamsource">AudioStreamSource</a> object providing the source for this track (if available). An <a class="internalDFN" href="#dfn-audiostreamsource">AudioStreamSource</a> + may be provided by a microphone, a peer connection, or a local audio file. Some <a class="internalDFN" href="#dfn-audiostreamtrack">AudioStreamTrack</a> sources may not expose + an <a class="internalDFN" href="#dfn-audiostreamsource">AudioStreamSource</a> object, in which case this property must return <code>null</code>. When an <a class="internalDFN" href="#dfn-audiostreamtrack">AudioStreamTrack</a> + is first created, and while it remains in the <code>"new"</code> state, the <code>source</code> attribute must return <code>null</code>. + </dd></dl></section> + </section> + </section> + </section> + + <section id="media-stream-sources"> + <!--OddPage--><h2><span class="secno">3. </span>Media Stream Sources</h2> + + <p><a class="internalDFN" href="#dfn-videostreamsource">VideoStreamSource</a> and <a class="internalDFN" href="#dfn-audiostreamsource">AudioStreamSource</a> objects are instantiated by the user agent to represent a source that is providing the + media for a <a class="internalDFN" href="#dfn-mediastreamtrack">MediaStreamTrack</a>. The association of a source object with a media track occurs asynchronously after + permission for use of the track has been requested by <code>getUserMedia</code>. When the user agent has attached + the source of a <a class="internalDFN" href="#dfn-mediastreamtrack">MediaStreamTrack</a>, the source object can be accessed via that track's <code>source</code> attribute. + </p> + + <div class="note"><div class="note-title"><span>Note</span></div><p><strong>Note: </strong> Some <a class="internalDFN" href="#dfn-mediastreamtrack">MediaStreamTrack</a>s may not provide a <code>source</code> object; for + example, if the source is coming from an encrypted media source, or a local file source. + </p></div> + + <div class="issue"><div class="issue-title"><span>Issue 3</span></div><p><strong>Issue: </strong> Need to define whether source objects are singletons. For example, if one track adds an expando + property onto a source object, will another track that has that same source see the expando on its source object? + </p></div> + + <section id="local-video-and-audio-sources"> + <h3><span class="secno">3.1 </span>Local Video and Audio Sources</h3> + + <p><a class="internalDFN" href="#dfn-videostreamsource">VideoStreamSource</a> and <a class="internalDFN" href="#dfn-audiostreamsource">AudioStreamSource</a> objects are created by the user agent to represent a camera or microphone + device/source for which the source attributes can be inspected and/or changed. At the moment these are limited to local cameras, + local microphones, and peer connection sources, but additional sources can be defined later (such a local file system sources + for images or audio files). + </p> + + <section id="videostreamsource-interface"> + <h4><span class="secno">3.1.1 </span><code><dfn id="dfn-videostreamsource">VideoStreamSource</dfn></code> interface</h4> + <pre class="idl"><span class="idlInterface" id="idl-def-VideoStreamSource">interface <span class="idlInterfaceID">VideoStreamSource</span> : <span class="idlSuperclass"><a>EventTarget</a></span> { +<span class="idlAttribute"> readonly attribute <span class="idlAttrType"><a>unsigned long</a></span> <span class="idlAttrName"><a href="#widl-VideoStreamSource-width">width</a></span>;</span> +<span class="idlAttribute"> readonly attribute <span class="idlAttrType"><a>unsigned long</a></span> <span class="idlAttrName"><a href="#widl-VideoStreamSource-height">height</a></span>;</span> +<span class="idlAttribute"> readonly attribute <span class="idlAttrType"><a>float</a></span> <span class="idlAttrName"><a href="#widl-VideoStreamSource-frameRate">frameRate</a></span>;</span> +<span class="idlAttribute"> readonly attribute <span class="idlAttrType"><a class="idlType" href="#idl-def-VideoRotationEnum"><code>VideoRotationEnum</code></a></span> <span class="idlAttrName"><a href="#widl-VideoStreamSource-rotation">rotation</a></span>;</span> +<span class="idlAttribute"> readonly attribute <span class="idlAttrType"><a class="idlType" href="#idl-def-VideoMirrorEnum"><code>VideoMirrorEnum</code></a></span> <span class="idlAttrName"><a href="#widl-VideoStreamSource-mirror">mirror</a></span>;</span> +<span class="idlAttribute"> readonly attribute <span class="idlAttrType"><a>float</a></span> <span class="idlAttrName"><a href="#widl-VideoStreamSource-zoom">zoom</a></span>;</span> +<span class="idlAttribute"> readonly attribute <span class="idlAttrType"><a class="idlType" href="#idl-def-VideoFocusModeEnum"><code>VideoFocusModeEnum</code></a></span> <span class="idlAttrName"><a href="#widl-VideoStreamSource-focusMode">focusMode</a></span>;</span> +<span class="idlAttribute"> readonly attribute <span class="idlAttrType"><a class="idlType" href="#idl-def-VideoFillLightModeEnum"><code>VideoFillLightModeEnum</code></a></span> <span class="idlAttrName"><a href="#widl-VideoStreamSource-fillLightMode">fillLightMode</a></span>;</span> +<span class="idlMethod"> <span class="idlMethType"><a>void</a></span> <span class="idlMethName"><a href="#widl-VideoStreamSource-stop-void">stop</a></span> ();</span> +<span class="idlMethod"> static <span class="idlMethType"><a>unsigned long</a></span> <span class="idlMethName"><a href="#widl-VideoStreamSource-getNumDevices-unsigned-long">getNumDevices</a></span> ();</span> +};</span></pre><section><h5 id="attributes-3">Attributes</h5><dl class="attributes"><dt id="widl-VideoStreamSource-width"><code>width</code> of type <span class="idlAttrType"><a>unsigned long</a></span>, readonly</dt><dd>The "natural" width (in pixels) of the source of the video flowing through the track. For cameras implementing this + interface, this value represents the current setting of the camera's sensor (in terms of number of pixels). This value is + independent of the camera's rotation (if the camera's rotation setting is changed, it does not impact this value). + For example, consider a camera setting with width of 1024 pixels and height of 768 pixels. If the camera's rotation + setting is changed by 90 degrees, the width is still reported as 1024 pixels. However, a <code><video></code> element sink used + to preview this track would report a width of 768 pixels (the effective width with rotation factored in). + </dd><dt id="widl-VideoStreamSource-height"><code>height</code> of type <span class="idlAttrType"><a>unsigned long</a></span>, readonly</dt><dd>The "natural" height (in pixels) of the video provided by this source. See the "width" attribute for additional info.</dd><dt id="widl-VideoStreamSource-frameRate"><code>frameRate</code> of type <span class="idlAttrType"><a>float</a></span>, readonly</dt><dd>The expected frames per second rate of video provided by this source.</dd><dt id="widl-VideoStreamSource-rotation"><code>rotation</code> of type <span class="idlAttrType"><a class="idlType" href="#idl-def-VideoRotationEnum"><code>VideoRotationEnum</code></a></span>, readonly</dt><dd>The current rotation value in use by the camera. If not supported, the property must be initialized to "0".</dd><dt id="widl-VideoStreamSource-mirror"><code>mirror</code> of type <span class="idlAttrType"><a class="idlType" href="#idl-def-VideoMirrorEnum"><code>VideoMirrorEnum</code></a></span>, readonly</dt><dd>he current image mirroring behavior being applied. If not supported, the property must be initialized to "none".</dd><dt id="widl-VideoStreamSource-zoom"><code>zoom</code> of type <span class="idlAttrType"><a>float</a></span>, readonly</dt><dd>The current zoom scale value in use by the camera. If not supported this property will always return 1.0.</dd><dt id="widl-VideoStreamSource-focusMode"><code>focusMode</code> of type <span class="idlAttrType"><a class="idlType" href="#idl-def-VideoFocusModeEnum"><code>VideoFocusModeEnum</code></a></span>, readonly</dt><dd>The camera's current focusMode state. The initial/default value is "auto".</dd><dt id="widl-VideoStreamSource-fillLightMode"><code>fillLightMode</code> of type <span class="idlAttrType"><a class="idlType" href="#idl-def-VideoFillLightModeEnum"><code>VideoFillLightModeEnum</code></a></span>, readonly</dt><dd>The camera's current fill light/flash mode.</dd></dl></section><section><h5 id="methods-1">Methods</h5><dl class="methods"><dt id="widl-VideoStremSource-stop-void"><code>stop</code></dt><dd>Stops this source, which will cause the related track to enter the <code>ended</code> state. Same behavior of the old LocalMediaStream's + stop API, but only affects this track source. + <div><em>No parameters.</em></div><div><em>Return type: </em><code><a>void</a></code></div></dd><dt id="widl-VideoStreamSource-getNumDevices-unsigned-long"><code>getNumDevices</code>, static</dt><dd>Returns the number of video sources that are currently available in this UA. As a static method, this information can be + queried without instantiating any <a class="internalDFN" href="#dfn-videostreamtrack">VideoStreamTrack</a> or <a class="internalDFN" href="#dfn-videostreamsource">VideoStreamSource</a> objects or without calling <code>getUserMedia</code>. + <div class="issue"><div class="issue-title"><span>Issue 4</span></div><p><strong>Issue: </strong> This information deliberately adds to the fingerprinting surface of the UA. However, this information + could also be obtained via other round-about techniques using <code>getUserMedia</code>. This editor deems it worthwhile directly providing + this data as it seems important for determining whether multiple devices of this type are available. + </p></div> + <div class="issue"><div class="issue-title"><span>Issue 5</span></div><p><strong>Issue: </strong> The ability to be notified when new devices become available has been dropped from this proposal + (it was available in v4 via the DeviceList object). + </p></div> + <div><em>No parameters.</em></div><div><em>Return type: </em><code><a>unsigned long</a></code></div></dd></dl></section> + </section> + + <section id="videofacingenum-enumeration"> + <h4><span class="secno">3.1.2 </span><dfn id="dfn-videofacingenum">VideoFacingEnum</dfn> enumeration</h4> + <pre class="idl"><span class="idlEnum" id="idl-def-VideoFacingEnum">enum <span class="idlEnumID">VideoFacingEnum</span> { + "<span class="idlEnumItem">unknown</span>", + "<span class="idlEnumItem">user</span>", + "<span class="idlEnumItem">environment</span>" +};</span></pre><table class="simple"><tr><th colspan="2">Enumeration description</th></tr><tr><td><code>unknown</code></td><td>The relative directionality of the camera cannot be determined by the user agent based on the hardware.</td></tr><tr><td><code>user</code></td><td>The camera is facing toward the user (a self-view camera).</td></tr><tr><td><code>environment</code></td><td>The camera is facing away from the user (viewing the environment).</td></tr></table> + </section> + + <section id="videorotationenum-enumeration"> + <h4><span class="secno">3.1.3 </span><dfn id="dfn-videorotationenum">VideoRotationEnum</dfn> enumeration</h4> + <pre class="idl"><span class="idlEnum" id="idl-def-VideoRotationEnum">enum <span class="idlEnumID">VideoRotationEnum</span> { + "<span class="idlEnumItem">0</span>", + "<span class="idlEnumItem">90</span>", + "<span class="idlEnumItem">180</span>", + "<span class="idlEnumItem">270</span>" +};</span></pre><table class="simple"><tr><th colspan="2">Enumeration description</th></tr><tr><td><code>0</code></td><td>No effective rotation applied (default value if no rotation is supported by the device software).</td></tr><tr><td><code>90</code></td><td>A rotation of 90 degrees counter-clockwise (270 degrees in a clockwise rotation).</td></tr><tr><td><code>180</code></td><td>A rotation of 180 degrees.</td></tr><tr><td><code>270</code></td><td>A rotation of 270 degrees counter-clockwise (90 degrees in a clockwise rotation).</td></tr></table> + </section> + + <section id="videomirrorenum-enumeration"> + <h4><span class="secno">3.1.4 </span><dfn id="dfn-videomirrorenum">VideoMirrorEnum</dfn> enumeration</h4> + <pre class="idl"><span class="idlEnum" id="idl-def-VideoMirrorEnum">enum <span class="idlEnumID">VideoMirrorEnum</span> { + "<span class="idlEnumItem">none</span>", + "<span class="idlEnumItem">horizontal</span>", + "<span class="idlEnumItem">vertical</span>" +};</span></pre><table class="simple"><tr><th colspan="2">Enumeration description</th></tr><tr><td><code>none</code></td><td>No effective mirroring is being applied (default value if no mirroring is supported by the device software).</td></tr><tr><td><code>horizontal</code></td><td>The image is mirrored along the camera's width value. This setting does not consider the camera's current + rotation, so if a 90 degree rotation was also applied to this source, then the "horizontal" mirroring would + appear to be a vertical mirroring in a given sink.</td></tr><tr><td><code>vertical</code></td><td>The image is mirrored along the camera's height value. This setting does not consider the camera's current + rotation, so if a 90 degree rotation was also applied to this source, then the "vertical" mirroring would + appear to be a horizontal mirroring in a given sink.</td></tr></table> + </section> + + <section id="videofocusmodeenum-enumeration"> + <h4><span class="secno">3.1.5 </span><dfn id="dfn-videofocusmodeenum">VideoFocusModeEnum</dfn> enumeration</h4> + <pre class="idl"><span class="idlEnum" id="idl-def-VideoFocusModeEnum">enum <span class="idlEnumID">VideoFocusModeEnum</span> { + "<span class="idlEnumItem">notavailable</span>", + "<span class="idlEnumItem">auto</span>", + "<span class="idlEnumItem">manual</span>" +};</span></pre><table class="simple"><tr><th colspan="2">Enumeration description</th></tr><tr><td><code>notavailable</code></td><td>This camera does not have an option to change focus modes.</td></tr><tr><td><code>auto</code></td><td>The camera auto-focuses.</td></tr><tr><td><code>manual</code></td><td>The camera must be manually focused.</td></tr></table> + </section> + + <section id="videofilllightmodeenum-enumeration"> + <h4><span class="secno">3.1.6 </span><dfn id="dfn-videofilllightmodeenum">VideoFillLightModeEnum</dfn> enumeration</h4> + <pre class="idl"><span class="idlEnum" id="idl-def-VideoFillLightModeEnum">enum <span class="idlEnumID">VideoFillLightModeEnum</span> { + "<span class="idlEnumItem">notavailable</span>", + "<span class="idlEnumItem">auto</span>", + "<span class="idlEnumItem">off</span>", + "<span class="idlEnumItem">flash</span>", + "<span class="idlEnumItem">on</span>" +};</span></pre><table class="simple"><tr><th colspan="2">Enumeration description</th></tr><tr><td><code>notavailable</code></td><td>This video device does not have an option to change fill light modes (e.g., the camera does not have a flash).</td></tr><tr><td><code>auto</code></td><td>The video device's fill light will be enabled when required (typically low light conditions). Otherwise it will be + off. Note that <code>auto</code> does not guarantee that a flash will fire when <code>takePicture</code> is called. + Use <code>flash</code> to guarantee firing of the flash for the <code>takePicture</code> API. <code>auto</code> is the initial value. + </td></tr><tr><td><code>off</code></td><td>The video device's fill light and/or flash will not be used.</td></tr><tr><td><code>flash</code></td><td>If the video device is a camera supporting high-resolution photo-mode, this setting will always cause the flash to fire + for the <code>takePicture</code> API. Otherwise, if the video device does not support this mode, this value is equivalent + to <code>auto</code>. + </td></tr><tr><td><code>on</code></td><td>The video device's fill light will be turned on (and remain on) until this setting is changed again, or the underlying track object + has ended. + </td></tr></table> + </section> + + <section id="audiostreamsource-interface"> + <h4><span class="secno">3.1.7 </span><code><dfn id="dfn-audiostreamsource">AudioStreamSource</dfn></code> interface</h4> + <pre class="idl"><span class="idlInterface" id="idl-def-AudioStreamSource">interface <span class="idlInterfaceID">AudioStreamSource</span> : <span class="idlSuperclass"><a>EventTarget</a></span> { +<span class="idlAttribute"> readonly attribute <span class="idlAttrType"><a>unsigned long</a></span> <span class="idlAttrName"><a href="#widl-AudioStreamSource-gain">gain</a></span>;</span> +<span class="idlMethod"> <span class="idlMethType"><a>void</a></span> <span class="idlMethName"><a href="#widl-AudioStreamSource-stop-void">stop</a></span> ();</span> +<span class="idlMethod"> static <span class="idlMethType"><a>unsigned long</a></span> <span class="idlMethName"><a href="#widl-AudioStreamSource-getNumDevices-unsigned-long">getNumDevices</a></span> ();</span> +};</span></pre><section><h5 id="attributes-4">Attributes</h5><dl class="attributes"><dt id="widl-AudioStreamSource-gain"><code>gain</code> of type <span class="idlAttrType"><a>unsigned long</a></span>, readonly</dt><dd>The sensitivity of the microphone. This value must be a whole number between 0 and 100 inclusive. + The gain value establishes the maximum threshold of the the microphone's sensitivity. When set to 0, + the microphone is essentially off (it will not be able to pick-up any sound). A value of 100 means + the microphone is configured for it's maximum gain/sensitivity. When first initialized for this + track, the gain value should be set to 50, the initial value. + </dd></dl></section><section><h5 id="methods-2">Methods</h5><dl class="methods"><dt id="widl-AudioStreamSource-stop-void"><code>stop</code></dt><dd>Causes this track to enter the <code>ended</code> state. Same behavior of the old LocalMediaStream's stop + API, but only for this track source. + <div><em>No parameters.</em></div><div><em>Return type: </em><code><a>void</a></code></div></dd><dt id="widl-AudioStreamSource-getNumDevices-unsigned-long"><code>getNumDevices</code>, static</dt><dd>Returns the number of potential audio sources that are available in this UA. As a static method, this information can be + queried without instantiating any <a class="internalDFN" href="#dfn-audiostreamtrack">AudioStreamTrack</a> or <a class="internalDFN" href="#dfn-audiostreamsource">AudioStreamSource</a> objects or without calling <code>getUserMedia</code>. + <div class="issue"><div class="issue-title"><span>Issue 6</span></div><p><strong>Issue: </strong> This information deliberately adds to the fingerprinting surface of the UA. However, this information + can also be obtained by other round-about techniques using <code>getUserMedia</code>, and is important for determining + whether multiple devices of this type are available. + </p></div> + <div class="issue"><div class="issue-title"><span>Issue 7</span></div><p><strong>Issue: </strong> The ability to be notified when new devices become available has been dropped from this proposal + (it was available in v4 via the DeviceList object). + </p></div> + <div><em>No parameters.</em></div><div><em>Return type: </em><code><a>unsigned long</a></code></div></dd></dl></section> + </section> + </section> + + <section id="camera-sources-with-high-resolution-picture-modes"> + <h3><span class="secno">3.2 </span>Camera sources with "high-resolution picture" modes</h3> + + <p>The PictureStreamSource derived interface is created by the user agent if the camera source providing the VideoStreamSource + supports an optional "high-resolution picture mode" with picture settings that are separate from those of + its basic video source (which is usually considered its <q>preview</q> mode). + </p> + + <p>The PictureStreamSource object presents a set of capabilities and controls for taking high-resolution pictures. The + unique settings of this object are only applied at the time when the takePicture API is invoked. + </p> + + <section id="picturestreamsource-interface"> + <h4><span class="secno">3.2.1 </span><code><dfn id="dfn-picturestreamsource">PictureStreamSource</dfn></code> interface</h4> + <pre class="idl"><span class="idlInterface" id="idl-def-PictureStreamSource">interface <span class="idlInterfaceID">PictureStreamSource</span> : <span class="idlSuperclass"><a class="idlType" href="#idl-def-VideoStreamSource"><code>VideoStreamSource</code></a></span> { +<span class="idlAttribute"> readonly attribute <span class="idlAttrType"><a>unsigned long</a></span> <span class="idlAttrName"><a href="#widl-PictureStreamSource-photoWidth">photoWidth</a></span>;</span> +<span class="idlAttribute"> readonly attribute <span class="idlAttrType"><a>unsigned long</a></span> <span class="idlAttrName"><a href="#widl-PictureStreamSource-photoHeight">photoHeight</a></span>;</span> +<span class="idlMethod"> <span class="idlMethType"><a>void</a></span> <span class="idlMethName"><a href="#widl-PictureStreamSource-takePicture-void">takePicture</a></span> ();</span> +<span class="idlAttribute"> attribute <span class="idlAttrType"><a>EventHandler</a></span> <span class="idlAttrName"><a href="#widl-PictureStreamSource-onpicture">onpicture</a></span>;</span> +<span class="idlAttribute"> attribute <span class="idlAttrType"><a>EventHandler</a></span> <span class="idlAttrName"><a href="#widl-PictureStreamSource-onpictureerror">onpictureerror</a></span>;</span> +};</span></pre><section><h5 id="attributes-5">Attributes</h5><dl class="attributes"><dt id="widl-PictureStreamSource-photoWidth"><code>photoWidth</code> of type <span class="idlAttrType"><a>unsigned long</a></span>, readonly</dt><dd>The width (in pixels) of the configured high-resolution photo-mode's sensor.</dd><dt id="widl-PictureStreamSource-photoHeight"><code>photoHeight</code> of type <span class="idlAttrType"><a>unsigned long</a></span>, readonly</dt><dd>The height (in pixels) of the configured high-resolution photo-mode's sensor.</dd><dt id="widl-PictureStreamSource-onpicture"><code>onpicture</code> of type <span class="idlAttrType"><a>EventHandler</a></span></dt><dd>Register/unregister for "picture" events. The handler should expect to get a BlobEvent object as its first + parameter. + <div class="note"><div class="note-title"><span>Note</span></div><p>The BlobEvent returns a picture (as a Blob) in a compressed format (for example: PNG/JPEG) rather than a + raw ImageData object due to the expected large, uncompressed size of the resulting pictures.</p></div> + <div class="issue"><div class="issue-title"><span>Issue 9</span></div><p>This Event type (BlobEvent) should be the same thing used in the recording proposal.</p></div> + </dd><dt id="widl-PictureStreamSource-onpictureerror"><code>onpictureerror</code> of type <span class="idlAttrType"><a>EventHandler</a></span></dt><dd>In the event of an error taking the picture, a "pictureerror" event will be dispatched instead of a "picture" event. + The "pictureerror" is a simple event of type Event. + </dd></dl></section><section><h5 id="methods-3">Methods</h5><dl class="methods"><dt id="widl-PictureStreamSource-takePicture-void"><code>takePicture</code></dt><dd>Temporarily (asynchronously) switches the camera into "high resolution picture mode", applies the settings that + are unique to this object to the stream (switches the width/height to those of the photoWidth/photoHeight), and + records/encodes an image (using a user-agent determined format) into a Blob object. + Finally, queues a task to fire a "picture" event containing the resulting picture Blob instance. + <div class="issue"><div class="issue-title"><span>Issue 8</span></div><p><strong>Issue: </strong> We could consider providing a hint or setting for the desired picture format?</p></div> + <div><em>No parameters.</em></div><div><em>Return type: </em><code><a>void</a></code></div></dd></dl></section> + </section> + + <section id="blobevent-interface"> + <h4><span class="secno">3.2.2 </span><code>BlobEvent</code> interface</h4> + <pre class="idl"><span class="idlInterface" id="idl-def-BlobEvent">[<span class="extAttr">Constructor(DOMString type, optional BlobEventInit blobInitDict)</span>] +interface <span class="idlInterfaceID">BlobEvent</span> : <span class="idlSuperclass"><a>Event</a></span> { +<span class="idlAttribute"> readonly attribute <span class="idlAttrType"><a>Blob</a></span> <span class="idlAttrName"><a href="#widl-BlobEvent-data">data</a></span>;</span> +};</span></pre><section><h5 id="attributes-6">Attributes</h5><dl class="attributes"><dt id="widl-BlobEvent-data"><code>data</code> of type <span class="idlAttrType"><a>Blob</a></span>, readonly</dt><dd>Returns a Blob object whose type attribute indicates the encoding of the blob data. An implementation must + return a Blob in a format that is capable of being viewed in an HTML <code><img></code> tag. + </dd></dl></section> + </section> + + <section id="blobeventinit-dictionary"> + <h4><span class="secno">3.2.3 </span>BlobEventInit dictionary</h4> + <pre class="idl"><span class="idlDictionary" id="idl-def-BlobEventInit">dictionary <span class="idlDictionaryID">BlobEventInit</span> : <span class="idlSuperclass"><a>EventInit</a></span> { +<span class="idlMember"> <span class="idlMemberType"><a>Blob</a></span> <span class="idlMemberName"><a href="#widl-BlobEventInit-data">data</a></span>;</span> +};</span></pre><section><h5 id="dictionary-blobeventinit-members">Dictionary <a class="idlType" href="#idl-def-BlobEventInit"><code>BlobEventInit</code></a> Members</h5><dl class="dictionary-members"><dt id="widl-BlobEventInit-data"><code>data</code> of type <span class="idlMemberType"><a>Blob</a></span></dt><dd>A Blob object containing the data to deliver via this event.</dd></dl></section> + </section> + </section> + + <section id="remote-media-sources"> + <h3><span class="secno">3.3 </span>Remote Media Sources</h3> + + <p>When MediaStreams are transmitted over the network by way of a peer connection, the tracks that are created + on the remote side of the MediaStream will have a remote media source attached as the track's source. This source object allows + remote-consumers of the MediaStream's tracks to request specific changes to the tracks. These change requests will be + serviced by the RTCPeerConnection source object which is streaming the media over the network. + </p> + + <section id="videostreamremotesource-interface"> + <h4><span class="secno">3.3.1 </span><code><dfn id="dfn-videostreamremotesource">VideoStreamRemoteSource</dfn></code> interface</h4> + <pre class="idl"><span class="idlInterface" id="idl-def-VideoStreamRemoteSource">interface <span class="idlInterfaceID">VideoStreamRemoteSource</span> : <span class="idlSuperclass"><a>EventTarget</a></span> { +<span class="idlAttribute"> readonly attribute <span class="idlAttrType"><a>unsigned long</a></span> <span class="idlAttrName"><a href="#widl-VideoStreamRemoteSource-width">width</a></span>;</span> +<span class="idlAttribute"> readonly attribute <span class="idlAttrType"><a>unsigned long</a></span> <span class="idlAttrName"><a href="#widl-VideoStreamRemoteSource-height">height</a></span>;</span> +<span class="idlAttribute"> readonly attribute <span class="idlAttrType"><a>float</a></span> <span class="idlAttrName"><a href="#widl-VideoStreamRemoteSource-frameRate">frameRate</a></span>;</span> +<span class="idlAttribute"> readonly attribute <span class="idlAttrType"><a>float</a></span> <span class="idlAttrName"><a href="#widl-VideoStreamRemoteSource-bitRate">bitRate</a></span>;</span> +};</span></pre><section><h5 id="attributes-7">Attributes</h5><dl class="attributes"><dt id="widl-VideoStreamRemoteSource-width"><code>width</code> of type <span class="idlAttrType"><a>unsigned long</a></span>, readonly</dt><dd>The current video transmission width.</dd><dt id="widl-VideoStreamRemoteSource-height"><code>height</code> of type <span class="idlAttrType"><a>unsigned long</a></span>, readonly</dt><dd>The current video transmission height.</dd><dt id="widl-VideoStreamRemoteSource-frameRate"><code>frameRate</code> of type <span class="idlAttrType"><a>float</a></span>, readonly</dt><dd>The current video frames-per-second.</dd><dt id="widl-VideoStreamRemoteSource-bitRate"><code>bitRate</code> of type <span class="idlAttrType"><a>float</a></span>, readonly</dt><dd>The current video bitRate.</dd></dl></section> + </section> + + <section id="audiostreamremotesource-interface"> + <h4><span class="secno">3.3.2 </span><code><dfn id="dfn-audiostreamremotesource">AudioStreamRemoteSource</dfn></code> interface</h4> + <pre class="idl"><span class="idlInterface" id="idl-def-AudioStreamRemoteSource">interface <span class="idlInterfaceID">AudioStreamRemoteSource</span> : <span class="idlSuperclass"><a>EventTarget</a></span> { +<span class="idlAttribute"> readonly attribute <span class="idlAttrType"><a>float</a></span> <span class="idlAttrName"><a href="#widl-AudioStreamRemoteSource-bitRate">bitRate</a></span>;</span> +};</span></pre><section><h5 id="attributes-8">Attributes</h5><dl class="attributes"><dt id="widl-AudioStreamRemoteSource-bitRate"><code>bitRate</code> of type <span class="idlAttrType"><a>float</a></span>, readonly</dt><dd>The current video bitRate.</dd></dl></section> + </section> + </section> + + <section id="other-settings-out-of-scope-in-this-proposal"> + <h3><span class="secno">3.4 </span>Other Settings (out-of-scope in this proposal)</h3> + + <p>The following settings have been proposed, but are not included in this version to keep the + initial set of settings scoped to those that: + </p> + + <ol> + <li>cannot be easily computed in post-processing</li> + <li>are not redundant with other settings</li> + <li>are settings found in nearly all devices (common)</li> + <li>can be easily tested for conformance</li> + </ol> + + <p>Each setting also includes a brief explanatory rationale for why it's not included:</p> + + <ol> + <li><code>horizontalAspectRatio</code> - easily calculated based on width/height in the dimension values</li> + <li><code>verticalAspectRatio</code> - see horizontalAspectRatio explanation</li> + <li><code>orientation</code> - can be easily calculated based on the width/height values and the current rotation</li> + <li><code>aperatureSize</code> - while more common on digital cameras, not particularly common on webcams (major use-case + for this feature)</li> + <li><code>shutterSpeed</code> - see aperatureSize explanation</li> + <li><code>denoise</code> - may require specification of the algorithm processing or related image processing filter required + to implement. + </li> + <li><code>effects</code> - sounds like a v2 or independent feature (depending on the effect).</li> + <li><code>faceDetection</code> - sounds like a v2 feature. Can also be done using post-processing techniques (though + perhaps not as fast...) + </li> + <li><code>antiShake</code> - sounds like a v2 feature.</li> + <li><code>geoTagging</code> - this can be independently associated with a recorded picture/video/audio clip using the + Geolocation API. Automatically hooking up Geolocation to Media Capture sounds like an exercise for v2 + given the possible complications. + </li> + <li><code>highDynamicRange</code> - not sure how this can be specified, or if this is just a v2 feature.</li> + <li><code>skintoneEnhancement</code> - not a particularly common setting.</li> + <li><code>shutterSound</code> - Can be accomplished by syncing custom audio playback via the <code><audio></code> tag if desired. + By default, there will be no sound issued. + </li> + <li><code>redEyeReduction</code> - photo-specific setting. (Could be considered if photo-specific settings + are introduced.) + </li> + <li><code>meteringMode</code> - photo-specific setting. (Could be considered if photo-specific settings + are introduced.)</li> + <li><code>iso</code> - photo-specific setting. while more common on digital cameras, not particularly common on webcams (major use-case + for this feature)</li> + <li><code>sceneMode</code> - while more common on digital cameras, not particularly common on webcams (major use-case + for this feature)</li> + <li><code>antiFlicker</code> - not a particularly common setting.</li> + <li><code>zeroShutterLag</code> - this seems more like a <em>hope</em> than a setting. I'd rather just have implementations + make the shutter snap as quickly as possible after takePicture, rather than requiring an opt-in/opt-out + for this setting. + </li> + </ol> + + <p>The following settings may be included by working group decision:</p> + + <ol> + <li>exposure</li> + <li>exposureCompensation (is this the same as exposure?)</li> + <li>autoExposureMode</li> + <li>brightness</li> + <li>contrast</li> + <li>saturation</li> + <li>sharpness</li> + <li>evShift</li> + <li>whiteBalance</li> + </ol> + </section> + </section> + + <section id="changing-stream-source-settings"> + <!--OddPage--><h2><span class="secno">4. </span>Changing Stream Source Settings</h2> + + <p>This proposal simplifies the application of settings over the previous proposal and unifies the setting names + with the constraint names and syntax. This unification allows developers to use the same syntax when defining + constraints as well as with the settings APIs. + </p> + + <p>The settings for each track (if available) are all conveniently located on the <code>source</code> object for the respective track. Each setting + is defined by a readonly attribute (for example the "width" attribute) which serves as a feature-detection for + the given setting, as well as the current value of the setting at any point in time. The constraint name for each + setting are the same as the name of the readonly attribute. For example, "photoWidth" is both the name of the setting + as well as the name of the constraint. All of the constraints defined in this proposal are listed later. + </p> + + <p>Reading the current settings are as simple as reading the readonly attribute of the same name. Each setting also has + a range of appropriate values (its capabilities), either enumerated values or a range continuum--these are the same ranges/enumerated + values that may be used when expressing constraints for the given setting. Retrieving the capabilities of a given setting + is done via a <code>getRange</code> API on each source object. Similarly, requesting a change to a setting is done via a + <code>set</code> API on each source object. Finally, for symmetry a <code>get</code> method is also defined which reports + the current value of any setting. + </p> + + <p>As noted in prior proposals, camera/microphone settings must be applied asynchronously to ensure that web + applications can remain responsive for all device types that may not respond quickly to setting changes. + This is especially true for settings communications over a peer connection. + </p> + + <section id="expectations-around-changing-settings"> + <h3><span class="secno">4.1 </span>Expectations around changing settings</h3> + + <p>Browsers provide a media pipeline from sources to sinks. In a browser, sinks are the <img>, <video> and <audio> tags. Traditional sources + include streamed content, files and web resources. The media produced by these sources typically does not change over time - these sources can be + considered to be static.</p> + + <p>The sinks that display these sources to the user (the actual tags themselves) have a variety of controls for manipulating the source content. For + example, an <img> tag scales down a huge source image of 1600x1200 pixels to fit in a rectangle defined with <code>width="400"</code> and + <code>height="300"</code>.</p> + + <p>The getUserMedia API adds dynamic sources such as microphones and cameras - the characteristics of these sources can change in response to application + needs. These sources can be considered to be dynamic in nature. A <video> element that displays media from a dynamic source can either perform + scaling or it can feed back information along the media pipeline and have the source produce content more suitable for display.</p> + + <div class="note"><div class="note-title"><span>Note</span></div><p><strong>Note: </strong> This sort of feedback loop is obviously just enabling an "optimization", but it's a non-trivial gain. This + optimization can save battery, allow for less network congestion, etc...</p></div> + + <p>This proposal assumes that <code>MediaStream</code> sinks (such as <code><video></code>, <code><audio></code>, + and even <code>RTCPeerConnection</code>) will continue to have mechanisms to further transform the source stream beyond that + which the settings described in this proposal offer. (The sink transformation options, including those of <code>RTCPeerConnection</code> + are outside the scope of this proposal.)</p> + + <p>The act of changing a setting on a stream's source will, by definition, affect all down-level sinks that are using that source. Many sinks may be able + to take these changes in stride, such as the <code><video></code> element or <code>RTCPeerConnection</code>. Others like the Recorder API may fail + as a result of a source change.</p> + + <p>The <code>RTCPeerConnection</code> is an interesting object because it acts simultaneously as both a sink <strong>and</strong> a source for over-the-network + streams. As a sink, it has source transformational capabilities (e.g., lowering bit-rates, scaling-up or down resolutions, adjusting frame-rates), and as a + source it may have its own settings changed by a track source that it provides (in this proposal, such sources are the <a class="internalDFN" href="#dfn-videostreamremotesource">VideoStreamRemoteSource</a> and + <a class="internalDFN" href="#dfn-audiostreamremotesource">AudioStreamRemoteSource</a> objects). + </p> + + <p>To illustrate how changes to a given source impact various sinks, consider the following example. This example only uses width and height, but the same + principles apply to any of the settings exposed in this proposal. In the first figure a home client has obtained a video source + from its local video camera. The source device's width and height are 800 pixels by 600 pixels, respectively. Three <code>MediaStream</code> objects on the + home client contain tracks that use this same source. The three media streams are connected to three different sinks, a <code><video></code> element (A), + another <code><video></code> element (B), and a peer connection (C). The peer connection is streaming the source video to an away client. On the away client + there are two media streams with tracks that use the peer connection as a source. These two media streams are connected to two <code><video></code> element + sinks (Y and Z). + </p> + + <img title="Changing media stream source effects: before the requested change" src="change_settings_before.png"> + + <p>Note that in the current state, all of the sinks on the home client must apply a transformation to the original source's dimensions. A is scaling the video up + (resulting in loss of quality), B is scaling the video down, and C is also scaling the video up slightly for sending over the network. On the away client, sink + Y is scaling the video way down, while sink Z is not applying any scaling. + </p> + + <p>Using the settings APIs defined in the next section, the home client's video source is changed to a higher resolution (1920 by 1200 pixels).</p> + + <img title="Changing media stream source effects: after the requested change" src="change_settings_after.png"> + + <p>Note that the source change immediately effects all of the sinks on home client, but does not impact any of the sinks (or sources) on the away client. With the + increase in the home client source video's dimensions, sink A no longer has to perform any scaling, while sink B must scale down even further than before. + Sink C (the peer connection) must now scale down the video in order to keep the transmission constant to the away client. + </p> + + <p>While not shown, an equally valid settings change request could be made of the away client video source (the peer connection on the away client's side). + This would not only impact sink Y and Z in the same manner as before, but would also cause re-negotiation with the peer connection on the home + client in order to alter the transformation that it is applying to the home client's video source. Such a change <strong>would not</strong> change anything + related to sink A or B or the home client's video source. + </p> + + <div class="note"><div class="note-title"><span>Note</span></div><p><strong>Note: </strong> This proposal does not define a mechanism by which a change to the away client's video source could + automatically trigger a change to the home client's video source. Implementations may choose to make such source-to-sink optimizations as long as they only + do so within the constraints established by the application, as the next example describes. + </p></div> + + <p>It is fairly obvious that changes to a given source will impact sink consumers. However, in some situations changes to a given sink may also be cause for + implementations to adjust the characteristics of a source's stream. This is illustrated in the following figures. In the first figure below, the home + client's video source is sending a video stream sized at 1920 by 1200 pixels. The video source is also unconstrained, such that the exact source dimensions + are flexible as far as the application is concerned. Two <code>MediaStream</code> objects contain tracks that use this same source, and those + <code>MediaStream</code>s are connected to two different <code><video></code> element sinks A and B. Sink A has been sized to <code>width="1920"</code> and + <code>height="1200"</code> and is displaying the sources video without any transformations. Sink B has been sized smaller and as a result, is scaling the + video down to fit its rectangle of 320 pixels across by 200 pixels down. + </p> + + <img title="Changing media stream sinks may affect sources: before the requested change" src="change_settings_before2.png"> + + <p>When the application changes sink A to a smaller dimension (from 1920 to 1024 pixels wide and from 1200 to 768 pixels tall), the browser's media pipeline may + recognize that none of its sinks require the higher source resolution, and needless work is being done both on the part of the source and on sink A. In + such a case and without any other constraints forcing the source to continue producing the higher resolution video, the media pipeline may change the source + resolution:</p> + + <img title="Changing media stream sinks may affect sources: after the requested change" src="change_settings_after2.png"> + + <p>In the above figure, the home client's video source resolution was changed to the max(sinkA, sinkB) in order to optimize playback. While not shown above, the + same behavior could apply to peer connections and other sinks.</p> + </section> + + <section id="streamsourcesettings-mix-in-interface"> + <h3><span class="secno">4.2 </span><code>StreamSourceSettings</code> mix-in interface</h3> + <pre><code><a class="internalDFN" href="#dfn-videostreamsource">VideoStreamSource</a></code> implements <code>StreamSourceSettings</code>;</pre> + <pre><code><a class="internalDFN" href="#dfn-audiostreamsource">AudioStreamSource</a></code> implements <code>StreamSourceSettings</code>;</pre> + <pre><code><a class="internalDFN" href="#dfn-videostreamremotesource">VideoStreamRemoteSource</a></code> implements <code>StreamSourceSettings</code>;</pre> + <pre><code><a class="internalDFN" href="#dfn-audiostreamremotesource">AudioStreamRemoteSource</a></code> implements <code>StreamSourceSettings</code>;</pre> + <pre class="idl"><span class="idlInterface" id="idl-def-StreamSourceSettings">[<span class="extAttr">NoInterfaceObject</span>] +interface <span class="idlInterfaceID">StreamSourceSettings</span> { +<span class="idlMethod"> <span class="idlMethType">(<a class="idlType" href="#idl-def-MediaSettingsRange"><code>MediaSettingsRange</code></a> or <a class="idlType" href="#idl-def-MediaSettingsList"><code>MediaSettingsList</code></a>)</span> <span class="idlMethName"><a href="#widl-StreamSourceSettings-getRange-MediaSettingsRange-MediaSettingsList-DOMString-settingName">getRange</a></span> (<span class="idlParam"><span class="idlParamType"><a>DOMString</a></span> <span class="idlParamName">settingName</span></span>);</span> +<span class="idlMethod"> <span class="idlMethType"><a>any</a></span> <span class="idlMethName"><a href="#widl-StreamSourceSettings-get-any-DOMString-settingName">get</a></span> (<span class="idlParam"><span class="idlParamType"><a>DOMString</a></span> <span class="idlParamName">settingName</span></span>);</span> +<span class="idlMethod"> <span class="idlMethType"><a>void</a></span> <span class="idlMethName"><a href="#widl-StreamSourceSettings-set-void-MediaTrackConstraint-setting-boolean-isMandatory---false">set</a></span> (<span class="idlParam"><span class="idlParamType"><a>MediaTrackConstraint</a></span> <span class="idlParamName">setting</span></span>, <span class="idlParam">optional <span class="idlParamType"><a>boolean isMandatory =</a></span> <span class="idlParamName">false</span></span>);</span> +};</span></pre><section id="methods"><h4><span class="secno">4.2.1 </span>Methods</h4><dl class="methods"><dt id="widl-StreamSourceSettings-getRange-MediaSettingsRange-MediaSettingsList-DOMString-settingName"><code>getRange</code></dt><dd> + + <p>Each setting has an appropriate range of values. These may be either value ranges (a continuum of values) or + enumerated values but not both. Value ranges include a min and max value, while enumerated values are provided + as a list of values. Both types of setting ranges include an "initial" value, which is the value that is expected + to be the source device's default value when it is acquired. + </p> + + <p>MediaSettingsRange objects are returned when a setting is not an enumerated type. This specification will indicate what + the range of values must be for each setting. Given that implementations + of various hardware may not exactly map to the same range, an implementation should make a reasonable attempt to + translate and scale the hardware's setting onto the mapping provided by this specification. If this is not possible due + to a hardware setting supporting (for example) fewer levels of granularity, then the implementation should make the device + settings min value reflect the min value reported in this specification, and the same for the max value. Then for values + in between the min and max, the implementation may round to the nearest supported value and report that value in the + setting. + </p> + + <div class="note"><div class="note-title"><span>Note</span></div><p>For example, if the setting is fluxCapacitance, and has a specified range from -10 (min) to 10 (max) in + this specification, but the implementation's fluxCapacitance hardware setting only supports values of "off" "medium" and + "full", then -10 should be mapped to "off", 10 should map to "full", and 0 should map to "medium". A request to change the + value to 3 should be rounded down to the closest supported setting (0). + </p></div> + + <p>MediaSettingsList objects should order their enumerated values from minimum to maximum where it makes sense, or in + the order defined by the enumerated type where applicable. + </p> + + <table class="simple"> + <thead> + <tr> + <th>Setting name</th> + <th>Dictionary return type</th> + <th>Notes</th> + </tr> + </thead> + <tbody> + <tr> + <td>width</td> + <td>MediaSettingsRange</td> + <td> + The range should span the video source's pre-set width values with min being the smallest width, and max the + largest width. The type of the min/max/initial values are unsigned long. + </td> + </tr> + <tr> + <td>photoWidth</td> + <td>MediaSettingsRange</td> + <td> + The range should span the video source's high-resolution photo-mode pre-set width values with min being the smallest width, and max the + largest width. The type of the min/max/initial values are unsigned long. + </td> + </tr> + <tr> + <td>height</td> + <td>MediaSettingsRange</td> + <td> + The range should span the video source's pre-set height values with min being the smallest width, and max the + largest width. The type of the min/max/initial values are unsigned long. + </td> + </tr> + <tr> + <td>photoHeight</td> + <td>MediaSettingsRange</td> + <td> + The range should span the video source's high-resolution photo-mode pre-set height values with min being the smallest width, and max the + largest width. The type of the min/max/initial values are unsigned long. + </td> + </tr> + <tr> + <td>frameRate</td> + <td>MediaSettingsRange</td> + <td> + The supported range of frame rates on the device. The type of the min/max/initial values are float. + </td> + </tr> + <tr> + <td>rotation</td> + <td>MediaSettingsList</td> + <td> + The available video rotation options on the source device. The type of the initial/values array is <a class="internalDFN" href="#dfn-videorotationenum">VideoRotationEnum</a> (DOMString). + </td> + </tr> + <tr> + <td>mirror</td> + <td>MediaSettingsList</td> + <td> + The available video mirror options on the source device. The type of the initial/values array is <a class="internalDFN" href="#dfn-videomirrorenum">VideoMirrorEnum</a> (DOMString). + </td> + </tr> + <tr> + <td>zoom</td> + <td>MediaSettingsRange</td> + <td> + The supported zoom range on the device. The type of the min/max/initial values are float. The initial value is 1. The float value is a scale + factor, for example 0.5 is zoomed out by double, while 2.0 is zoomed in by double. Requests should be rounded to the nearest supporting zoom + factor by the implementation (when zoom is supported). + </td> + </tr> + <tr> + <td>focusMode</td> + <td>MediaSettingsList</td> + <td> + The available focus mode options on the source device. The type of the initial/values array is <a class="internalDFN" href="#dfn-videofocusmodeenum">VideoFocusModeEnum</a> (DOMString). + </td> + </tr> + <tr> + <td>fillLightMode</td> + <td>MediaSettingsList</td> + <td> + The available fill light mode options on the source device. The type of the initial/values array is <a class="internalDFN" href="#dfn-videofilllightmodeenum">VideoFillLightModeEnum</a> (DOMString). + </td> + </tr> + <tr> + <td>gain</td> + <td>MediaSettingsRange</td> + <td> + The supported gain range on the device. The type of the min/max/initial values are unsigned long. The initial value is 50. + </td> + </tr> + <tr> + <td>bitRate</td> + <td>MediaSettingsRange</td> + <td> + The supported bit rate range on the device. The type of the min/max/initial values are float. + </td> + </tr> + </tbody> + </table> + <table class="parameters"><tr><th>Parameter</th><th>Type</th><th>Nullable</th><th>Optional</th><th>Description</th></tr><tr><td class="prmName">settingName</td><td class="prmType"><code><a>DOMString</a></code></td><td class="prmNullFalse">?</td><td class="prmOptFalse">?</td><td class="prmDesc">The name of the setting for which the range of expected values should be returned</td></tr></table><div><em>Return type: </em><code>(<a class="idlType" href="#idl-def-MediaSettingsRange"><code>MediaSettingsRange</code></a> or <a class="idlType" href="#idl-def-MediaSettingsList"><code>MediaSettingsList</code></a>)</code></div></dd><dt id="widl-StreamSourceSettings-get-any-DOMString-settingName"><code>get</code></dt><dd> + + Returns the current value of a given setting. This is equivalent to reading the IDL attribute of the same name on the source object. + <table class="parameters"><tr><th>Parameter</th><th>Type</th><th>Nullable</th><th>Optional</th><th>Description</th></tr><tr><td class="prmName">settingName</td><td class="prmType"><code><a>DOMString</a></code></td><td class="prmNullFalse">?</td><td class="prmOptFalse">?</td><td class="prmDesc">The name of the setting for which the current value of that setting should be returned</td></tr></table><div><em>Return type: </em><code><a>any</a></code></div></dd><dt id="widl-StreamSourceSettings-set-void-MediaTrackConstraint-setting-boolean-isMandatory---false"><code>set</code></dt><dd> + + <p>The <code>set</code> API is the mechanism for asynchronously requesting that the source device change the value + of a given setting. The API mirrors the syntax used for applying constraints. Generally, the <code>set</code> API + will be used to apply specific values to a setting (such as setting the <code>flashMode</code> setting to a specific + value), however ranges can also be applied using the same min/max syntax used in constraints (i.e., setting + <code>width</code> to a range between 800 and 1200 pixels). + </p> + + <p>The <code>set</code> API queues requests until the conclusion of the micro-task after which all of the settings requests + will be evaluated according to the constraint algorithm, and requests that can be honored will be applied to the + source device. Any requests specified using the <code>mandatory</code> parameter that could not be applied must + generate a <a>settingserror</a> event. All other non-mandatory requests that could not be applied do not cause + any notification to be generated. + </p> + + <p>For all of the given settings that were changed as a result of a sequence of calls to the <code>set</code> API during a + micro-task, one single <a>settingschanged</a> event will be generated containing the names of the settings that + changed.</p> + + <div class="note"><div class="note-title"><span>Note</span></div><p><strong>Example: </strong>To change the video source's dimensions to any aspect ratio where the height + is 768 pixels and the width is at least 300 pixels, would require two calls to <code>set</code>: <br> + <span style="white-space: pre;"><code>set({ width: { min: 300}}, true);<br>set({ height: 768}, true);</code></span> + </p></div> + + <p>In each case where the setting/constraint does not take an enumerated value, the implementation should attempt to match + the value onto the nearest supported value of the source device unless the mandatory flag is provided. In the case of + mandatory requests, if the setting cannot be exactly supported as requested, then the setting must fail and generate + a settingserror event. Regarding width/height values--if an implementation is able to scale the source video to + match the requested mandatory constraints, this need not cause a <a>settingserror</a> (but the result may be weirdly proportioned video). + </p> + <table class="parameters"><tr><th>Parameter</th><th>Type</th><th>Nullable</th><th>Optional</th><th>Description</th></tr><tr><td class="prmName">setting</td><td class="prmType"><code><a>MediaTrackConstraint</a></code></td><td class="prmNullFalse">?</td><td class="prmOptFalse">?</td><td class="prmDesc">A JavaScript object (dictionary) consisting of a single property which is the setting name to change, + and whose value is either a primitive value (float/DOMString/etc), or another dictionary consisting of a <code>min</code> + and/or <code>max</code> property and associated values.</td></tr><tr><td class="prmName">false</td><td class="prmType"><code><a>boolean isMandatory =</a></code></td><td class="prmNullFalse">?</td><td class="prmOptTrue">?</td><td class="prmDesc">A flag indicating whether this settings change request should be considered mandatory. If a value + of <code>true</code> is provided, then should the settings change fail for some reason, a <code><a>settingserror</a></code> + event will be raised. Otherwise, only <code><a>settingschanged</a></code> event will be dispatched for the settings + that were successfully changed. The default, if this flag is not provided, is <code>false</code></td></tr></table><div><em>Return type: </em><code><a>void</a></code></div></dd></dl></section> + + <section id="mediasettingsrange-dictionary"> + <h4><span class="secno">4.2.2 </span>MediaSettingsRange dictionary</h4> + <pre class="idl"><span class="idlDictionary" id="idl-def-MediaSettingsRange">dictionary <span class="idlDictionaryID">MediaSettingsRange</span> { +<span class="idlMember"> <span class="idlMemberType"><a>any</a></span> <span class="idlMemberName"><a href="#widl-MediaSettingsRange-max">max</a></span>;</span> +<span class="idlMember"> <span class="idlMemberType"><a>any</a></span> <span class="idlMemberName"><a href="#widl-MediaSettingsRange-min">min</a></span>;</span> +<span class="idlMember"> <span class="idlMemberType"><a>any</a></span> <span class="idlMemberName"><a href="#widl-MediaSettingsRange-initial">initial</a></span>;</span> +};</span></pre><section><h5 id="dictionary-mediasettingsrange-members">Dictionary <a class="idlType" href="#idl-def-MediaSettingsRange"><code>MediaSettingsRange</code></a> Members</h5><dl class="dictionary-members"><dt id="widl-MediaSettingsRange-max"><code>max</code> of type <span class="idlMemberType"><a>any</a></span></dt><dd>The maximum value of this setting. + <p>The type of this value is specific to the setting. Each setting will + describe a specific type. That type must be returned for this attribute.</p> + </dd><dt id="widl-MediaSettingsRange-min"><code>min</code> of type <span class="idlMemberType"><a>any</a></span></dt><dd>The minimum value of this setting. + <p>The type of this value is specific to the setting. Each setting will + describe a specific type. That type must be returned for this attribute.</p> + </dd><dt id="widl-MediaSettingsRange-initial"><code>initial</code> of type <span class="idlMemberType"><a>any</a></span></dt><dd>The initial value of this setting. When the object associated with this setting is first made available + to the application, the current value of the setting should be set to the initial value. + For example, in a browsing scenario, if one web site changes this setting and a subsequent web site + gets access to this same setting, the setting should have been reset back to its initial value. + <p>The type of this value is specific to the setting. Each setting will describe a specific type. That + type must be returned for this attribute.</p> + </dd></dl></section> + </section> + + <section id="mediasettingslist-dictionary"> + <h4><span class="secno">4.2.3 </span>MediaSettingsList dictionary</h4> + <pre class="idl"><span class="idlDictionary" id="idl-def-MediaSettingsList">dictionary <span class="idlDictionaryID">MediaSettingsList</span> { +<span class="idlMember"> <span class="idlMemberType">sequence<<a>any</a>></span> <span class="idlMemberName"><a href="#widl-MediaSettingsList-values">values</a></span>;</span> +<span class="idlMember"> <span class="idlMemberType"><a>any</a></span> <span class="idlMemberName"><a href="#widl-MediaSettingsList-initial">initial</a></span>;</span> +};</span></pre><section><h5 id="dictionary-mediasettingslist-members">Dictionary <a class="idlType" href="#idl-def-MediaSettingsList"><code>MediaSettingsList</code></a> Members</h5><dl class="dictionary-members"><dt id="widl-MediaSettingsList-values"><code>values</code> of type <span class="idlMemberType">sequence<<a>any</a>></span></dt><dd>An array of the values of the enumerated type for this setting. Items should be sorted + from min (at index 0) to max where applicable, or in the order listed in the enumerated type otherwise. + <p>The type of this value is specific to the setting. Each setting will describe a specific type. That + type must be returned for this attribute.</p> + </dd><dt id="widl-MediaSettingsList-initial"><code>initial</code> of type <span class="idlMemberType"><a>any</a></span></dt><dd>The initial value of this setting. When the object associated with this setting is first made available + to the application, the current value of the setting should be set to the initial value. + For example, in a browsing scenario, if one web site changes this setting and a subsequent web site + gets access to this same setting, the setting should have been reset back to its initial value. + <p>The type of this value is specific to the setting. Each setting will describe a specific type. That + type must be returned for this attribute.</p> + </dd></dl></section> + </section> + </section> + + <section id="tracking-the-result-of-constraint-application"> + <h3><span class="secno">4.3 </span>Tracking the result of constraint application</h3> + + <section id="mediasettingseventhandlers-mix-in-interface"> + <h4><span class="secno">4.3.1 </span><code>MediaSettingsEventHandlers</code> mix-in interface</h4> + <pre><code><a class="internalDFN" href="#dfn-audiostreamsource">AudioStreamSource</a></code> implements <code>MediaSettingsEventHandlers</code>;</pre> + <pre><code><a class="internalDFN" href="#dfn-videostreamsource">VideoStreamSource</a></code> implements <code>MediaSettingsEventHandlers</code>;</pre> + <pre><code><a class="internalDFN" href="#dfn-audiostreamremotesource">AudioStreamRemoteSource</a></code> implements <code>MediaSettingsEventHandlers</code>;</pre> + <pre><code><a class="internalDFN" href="#dfn-videostreamremotesource">VideoStreamRemoteSource</a></code> implements <code>MediaSettingsEventHandlers</code>;</pre> + <pre class="idl"><span class="idlInterface" id="idl-def-MediaSettingsEventHandlers">[<span class="extAttr">NoInterfaceObject</span>] +interface <span class="idlInterfaceID">MediaSettingsEventHandlers</span> { +<span class="idlAttribute"> attribute <span class="idlAttrType"><a>EventHandler</a></span> <span class="idlAttrName"><a href="#widl-MediaSettingsEventHandlers-onsettingserror">onsettingserror</a></span>;</span> +<span class="idlAttribute"> attribute <span class="idlAttrType"><a>EventHandler</a></span> <span class="idlAttrName"><a href="#widl-MediaSettingsEventHandlers-onsettingschanged">onsettingschanged</a></span>;</span> +};</span></pre><section><h5 id="attributes-9">Attributes</h5><dl class="attributes"><dt id="widl-MediaSettingsEventHandlers-onsettingserror"><code>onsettingserror</code> of type <span class="idlAttrType"><a>EventHandler</a></span></dt><dd>Register/unregister for "settingserror" events. The handler should expect to get a <a class="internalDFN" href="#dfn-mediasettingsevent">MediaSettingsEvent</a> object as its first + parameter. The event is fired asynchronously after settings change requests (using the <code>set</code> API have been made with at least + one such request using the mandatory flag. The MediaSettingsEvent reports the name of the settings that could not be + applied. The "settingschanged" event fires before the "settingserror" event (if any).</dd><dt id="widl-MediaSettingsEventHandlers-onsettingschanged"><code>onsettingschanged</code> of type <span class="idlAttrType"><a>EventHandler</a></span></dt><dd>Register/unregister for "settingschanged" events. The handler should expect to get a <a class="internalDFN" href="#dfn-mediasettingsevent">MediaSettingsEvent</a> object as its first + parameter. The event is fired asynchronously after the settings change requests are made and the settings have actually changed. + The "settingschanged" event fires before the "settingserror" event (if any). + </dd></dl></section> + </section> + + <section id="mediasettingsevent-interface"> + <h4><span class="secno">4.3.2 </span><code><dfn id="dfn-mediasettingsevent">MediaSettingsEvent</dfn></code> interface</h4> + <pre class="idl"><span class="idlInterface" id="idl-def-MediaSettingsEvent">[<span class="extAttr">Constructor(DOMString type, optional MediaSettingsEventInit eventInitDict)</span>] +interface <span class="idlInterfaceID">MediaSettingsEvent</span> : <span class="idlSuperclass"><a>Event</a></span> { +<span class="idlAttribute"> readonly attribute <span class="idlAttrType"><a>DOMString</a>[]</span> <span class="idlAttrName"><a href="#widl-MediaSettingsEvent-settings">settings</a></span>;</span> +};</span></pre><section><h5 id="attributes-10">Attributes</h5><dl class="attributes"><dt id="widl-MediaSettingsEvent-settings"><code>settings</code> of type array of <span class="idlAttrType"><a>DOMString</a></span>, readonly</dt><dd>A list of settings that failed or succeeded (depending on the event type).</dd></dl></section> + </section> + + <section id="mediasettingseventinit-dictionary"> + <h4><span class="secno">4.3.3 </span><code><dfn id="dfn-mediasettingseventinit">MediaSettingsEventInit</dfn></code> dictionary</h4> + <pre class="idl"><span class="idlDictionary" id="idl-def-MediaSettingsEventInit">dictionary <span class="idlDictionaryID">MediaSettingsEventInit</span> : <span class="idlSuperclass"><a>EventInit</a></span> { +<span class="idlMember"> <span class="idlMemberType">sequence<<a>DOMString</a>></span> <span class="idlMemberName"><a href="#widl-MediaSettingsEventInit-settings">settings</a></span>;</span> +};</span></pre><section><h5 id="dictionary-mediasettingseventinit-members">Dictionary <a class="idlType" href="#idl-def-MediaSettingsEventInit"><code>MediaSettingsEventInit</code></a> Members</h5><dl class="dictionary-members"><dt id="widl-MediaSettingsEventInit-settings"><code>settings</code> of type <span class="idlMemberType">sequence<<a>DOMString</a>></span></dt><dd>List of settings to populate into the MediaSettingsEvent object's settings readonly attribute.</dd></dl></section> + </section> + </section> + </section> + + <section id="constraints-defined-in-this-proposal"> + <!--OddPage--><h2><span class="secno">5. </span>Constraints Defined in this Proposal</h2> + + <p>This proposal defines several constraints for use with video and audio devices.</p> + + <section id="video-constraints"> + <h3><span class="secno">5.1 </span>Video Constraints</h3> + + <p>The following constraints are applicable to video devices</p> + + <section id="videoconstraints-dictionary"> + <h4><span class="secno">5.1.1 </span>VideoConstraints dictionary</h4> + <pre class="idl"><span class="idlDictionary" id="idl-def-VideoConstraints">dictionary <span class="idlDictionaryID">VideoConstraints</span> : <span class="idlSuperclass"><a>MediaTrackConstraintSet</a></span> { +<span class="idlMember"> <span class="idlMemberType">(<a>unsigned long</a> or <a class="idlType" href="#idl-def-MinMaxULongSubConstraint"><code>MinMaxULongSubConstraint</code></a>)</span> <span class="idlMemberName"><a href="#widl-VideoConstraints-width">width</a></span>;</span> +<span class="idlMember"> <span class="idlMemberType">(<a>unsigned long</a> or <a class="idlType" href="#idl-def-MinMaxULongSubConstraint"><code>MinMaxULongSubConstraint</code></a>)</span> <span class="idlMemberName"><a href="#widl-VideoConstraints-height">height</a></span>;</span> +<span class="idlMember"> <span class="idlMemberType">(<a>unsigned long</a> or <a class="idlType" href="#idl-def-MinMaxULongSubConstraint"><code>MinMaxULongSubConstraint</code></a>)</span> <span class="idlMemberName"><a href="#widl-VideoConstraints-photoWidth">photoWidth</a></span>;</span> +<span class="idlMember"> <span class="idlMemberType">(<a>unsigned long</a> or <a class="idlType" href="#idl-def-MinMaxULongSubConstraint"><code>MinMaxULongSubConstraint</code></a>)</span> <span class="idlMemberName"><a href="#widl-VideoConstraints-photoHeight">photoHeight</a></span>;</span> +<span class="idlMember"> <span class="idlMemberType"><a class="idlType" href="#idl-def-VideoRotationEnum"><code>VideoRotationEnum</code></a></span> <span class="idlMemberName"><a href="#widl-VideoConstraints-rotation">rotation</a></span>;</span> +<span class="idlMember"> <span class="idlMemberType"><a class="idlType" href="#idl-def-VideoMirrorEnum"><code>VideoMirrorEnum</code></a></span> <span class="idlMemberName"><a href="#widl-VideoConstraints-mirror">mirror</a></span>;</span> +<span class="idlMember"> <span class="idlMemberType">(<a>float</a> or <a class="idlType" href="#idl-def-MinMaxFloatSubConstraint"><code>MinMaxFloatSubConstraint</code></a>)</span> <span class="idlMemberName"><a href="#widl-VideoConstraints-zoom">zoom</a></span>;</span> +<span class="idlMember"> <span class="idlMemberType"><a class="idlType" href="#idl-def-VideoFocusModeEnum"><code>VideoFocusModeEnum</code></a></span> <span class="idlMemberName"><a href="#widl-VideoConstraints-focusMode">focusMode</a></span>;</span> +<span class="idlMember"> <span class="idlMemberType"><a class="idlType" href="#idl-def-VideoFillLightModeEnum"><code>VideoFillLightModeEnum</code></a></span> <span class="idlMemberName"><a href="#widl-VideoConstraints-fillLightMode">fillLightMode</a></span>;</span> +<span class="idlMember"> <span class="idlMemberType">(<a>float</a> or <a class="idlType" href="#idl-def-MinMaxFloatSubConstraint"><code>MinMaxFloatSubConstraint</code></a>)</span> <span class="idlMemberName"><a href="#widl-VideoConstraints-frameRate">frameRate</a></span>;</span> +<span class="idlMember"> <span class="idlMemberType">(<a>float</a> or <a class="idlType" href="#idl-def-MinMaxFloatSubConstraint"><code>MinMaxFloatSubConstraint</code></a>)</span> <span class="idlMemberName"><a href="#widl-VideoConstraints-bitRate">bitRate</a></span>;</span> +};</span></pre><section><h5 id="dictionary-videoconstraints-members">Dictionary <a class="idlType" href="#idl-def-VideoConstraints"><code>VideoConstraints</code></a> Members</h5><dl class="dictionary-members"><dt id="widl-VideoConstraints-width"><code>width</code> of type <span class="idlMemberType"><a>unsigned longMinMaxULongSubConstraint</a></span></dt><dd>A device that supports the desired width or width range.</dd><dt id="widl-VideoConstraints-height"><code>height</code> of type <span class="idlMemberType"><a>unsigned longMinMaxULongSubConstraint</a></span></dt><dd>A device that supports the desired height or height range.</dd><dt id="widl-VideoConstraints-photoWidth"><code>photoWidth</code> of type <span class="idlMemberType"><a>unsigned longMinMaxULongSubConstraint</a></span></dt><dd>A device that supports the desired width or width range for high-resolution photo-modes.</dd><dt id="widl-VideoConstraints-photoHeight"><code>photoHeight</code> of type <span class="idlMemberType"><a>unsigned longMinMaxUongSubConstraint</a></span></dt><dd>A device that supports the desired height or height range for high-resolution photo-modes.</dd><dt id="widl-VideoConstraints-rotation"><code>rotation</code> of type <span class="idlMemberType"><a class="idlType" href="#idl-def-VideoRotationEnum"><code>VideoRotationEnum</code></a></span></dt><dd>A device that supports the desired rotation.</dd><dt id="widl-VideoConstraints-mirror"><code>mirror</code> of type <span class="idlMemberType"><a class="idlType" href="#idl-def-VideoMirrorEnum"><code>VideoMirrorEnum</code></a></span></dt><dd>A device that supports the desired mirroring.</dd><dt id="widl-VideoConstraints-zoom"><code>zoom</code> of type <span class="idlMemberType"><a>floatMinMaxFloatSubConstraint</a></span></dt><dd>A device that supports the desired zoom setting.</dd><dt id="widl-VideoConstraints-focusMode"><code>focusMode</code> of type <span class="idlMemberType"><a class="idlType" href="#idl-def-VideoFocusModeEnum"><code>VideoFocusModeEnum</code></a></span></dt><d>A device that supports the desired focus mode.</dd><dt id="widl-VideoConstraints-fillLightMode"><code>fillLightMode</code> of type <span class="idlMemberType"><a class="idlType" href="#idl-def-VideoFillLightModeEnum"><code>VideoFillLightModeEnum</code></a></span></dt><dd>A device that supports the desired fill light (flash) mode.</dd><dt id="widl-VideoConstraints-frameRate"><code>frameRate</code> of type <span class="idlMemberType"><a>floatMinMaxFloatSubConstraint</a></span></dt><dd>A device that supports the desired frames per second.</dd><dt id="widl-VideoConstraints-bitRate"><code>bitRate</code> of type <span class="idlMemberType"><a>floatMinMaxFloatSubConstraint</a></span></dt><dd>A device that supports the desired bit rate.</dd></dl></section> + </section> + </section> + + <section id="audio-constraints"> + <h3><span class="secno">5.2 </span>Audio Constraints</h3> + + <p>The following constraints are applicable to audio devices</p> + + <section id="audioconstraints-dictionary"> + <h4><span class="secno">5.2.1 </span>AudioConstraints dictionary</h4> + <pre class="idl"><span class="idlDictionary" id="idl-def-AudioConstraints">dictionary <span class="idlDictionaryID">AudioConstraints</span> : <span class="idlSuperclass"><a>MediaTrackConstraintSet</a></span> { +<span class="idlMember"> <span class="idlMemberType">(<a>unsigned long</a> or <a class="idlType" href="#idl-def-MinMaxULongSubConstraint"><code>MinMaxULongSubConstraint</code></a>)</span> <span class="idlMemberName"><a href="#widl-AudioConstraints-gain">gain</a></span>;</span> +};</span></pre><section><h5 id="dictionary-audioconstraints-members">Dictionary <a class="idlType" href="#idl-def-AudioConstraints"><code>AudioConstraints</code></a> Members</h5><dl class="dictionary-members"><dt id="widl-AudioConstraints-gain"><code>gain</code> of type <span class="idlMemberType"><a>unsigned longMinMaxULongSubConstraint</a></span></dt><dd>A device that supports the desired gain or gain range.</dd></dl></section> + </section> + + </section> + + <section id="common-sub-constraint-structures"> + <h3><span class="secno">5.3 </span>Common sub-constraint structures</h3> + + <section id="minmaxulongsubconstraint-dictionary"> + <h4><span class="secno">5.3.1 </span><code><dfn id="dfn-minmaxulongsubconstraint">MinMaxULongSubConstraint</dfn></code> dictionary</h4> + <pre class="idl"><span class="idlDictionary" id="idl-def-MinMaxULongSubConstraint">dictionary <span class="idlDictionaryID">MinMaxULongSubConstraint</span> { +<span class="idlMember"> <span class="idlMemberType"><a>unsigned long</a></span> <span class="idlMemberName"><a href="#widl-MinMaxULongSubConstraint-max">max</a></span>;</span> +<span class="idlMember"> <span class="idlMemberType"><a>unsigned long</a></span> <span class="idlMemberName"><a href="#widl-MinMaxULongSubConstraint-min">min</a></span>;</span> +};</span></pre><section><h5 id="dictionary-minmaxulongsubconstraint-members">Dictionary <a class="idlType" href="#idl-def-MinMaxULongSubConstraint"><code>MinMaxULongSubConstraint</code></a> Members</h5><dl class="dictionary-members"><dt id="widl-MinMaxULongSubConstraint-max"><code>max</code> of type <span class="idlMemberType"><a>unsigned long</a></span></dt><dd>unsigned long min</dd><dt id="widl-MinMaxULongSubConstraint-min"><code>min</code> of type <span class="idlMemberType"><a>unsigned long</a></span></dt><dd></dd></dl></section> + </section> + + <section id="minmaxfloatsubconstraint-dictionary"> + <h4><span class="secno">5.3.2 </span><code><dfn id="dfn-minmaxfloatsubconstraint">MinMaxFloatSubConstraint</dfn></code> dictionary</h4> + <pre class="idl"><span class="idlDictionary" id="idl-def-MinMaxFloatSubConstraint">dictionary <span class="idlDictionaryID">MinMaxFloatSubConstraint</span> { +<span class="idlMember"> <span class="idlMemberType"><a>float</a></span> <span class="idlMemberName"><a href="#widl-MinMaxFloatSubConstraint-max">max</a></span>;</span> +<span class="idlMember"> <span class="idlMemberType"><a>float</a></span> <span class="idlMemberName"><a href="#widl-MinMaxFloatSubConstraint-min">min</a></span>;</span> +};</span></pre><section><h5 id="dictionary-minmaxfloatsubconstraint-members">Dictionary <a class="idlType" href="#idl-def-MinMaxFloatSubConstraint"><code>MinMaxFloatSubConstraint</code></a> Members</h5><dl class="dictionary-members"><dt id="widl-MinMaxFloatSubConstraint-max"><code>max</code> of type <span class="idlMemberType"><a>float</a></span></dt><dd>float min</dd><dt id="widl-MinMaxFloatSubConstraint-min"><code>min</code> of type <span class="idlMemberType"><a>float</a></span></dt><dd></dd></dl></section> + </section> + + </section> + </section> + + <section id="example-usage-scenarios"> + <!--OddPage--><h2><span class="secno">6. </span>Example usage scenarios</h2> + + <p>The following JavaScript examples demonstrate how the Settings APIs defined in this proposal could be used.</p> + + <section id="getting-access-to-a-video-and-or-audio-device-if-available"> + <h3><span class="secno">6.1 </span>Getting access to a video and/or audio device (if available)</h3> + + <pre>var audioTrack = (AudioStreamSource.getNumDevices() > 0) ? new AudioStreamTrack() : null; +if (audioTrack) + audioTrack.onstart = mediaStarted; +var videoTrack = (VideoStreamSource.getNumDevices() > 0) ? new VideoStreamTrack() : null; +if (videoTrack) + videoTrack.onstart = mediaStarted; +var MS = new MediaStream(); +MS.addTrack(audioTrack); +MS.addTrack(videoTrack); +navigator.getUserMedia(MS); + +function mediaStarted() { + // One of the video/audio devices started. +} +</pre> + </section> + + <section id="previewing-the-local-video-audio-in-html5-video-tag----scenario-is-unchanged"> + <h3><span class="secno">6.2 </span>Previewing the local video/audio in HTML5 video tag -- scenario is unchanged</h3> + + <pre>function mediaStarted() { + // objectURL technique + document.querySelector("video").src = URL.createObjectURL(MS, { autoRevoke: true }); // autoRevoke is the default + // direct-assign technique + document.querySelector("video").srcObject = MS; // Proposed API at this time +} +</pre> + </section> + + <section id="applying-resolution-constraints"> + <h3><span class="secno">6.3 </span>Applying resolution constraints</h3> + + <pre>function mediaStarted() { + var videoDevice = videoTrack.source; + var maxWidth = videoDevice.getRange("width").max; + var maxHeight = videoDevice.getRange("height").max; + // Check for 1080p+ support + if ((maxWidth >= 1920) && (maxHeight >= 1080)) { + // See if I need to change the current settings... + if ((videoDevice.width < 1920) && (videoDevice.height < 1080)) { + videoDevice.set({ width: maxWidth}, true); + videoDevice.set({ height: maxHeight}, true); + videoDevice.onsettingserror = failureToComply; + } + } + else + failureToComply(); +} + +function failureToComply(e) { + if (e) + console.error("Devices failed to change " + e.settings); // 'width' and/or 'height' + else + console.error("Device doesn't support at least 1080p"); +} +</pre> + </section> + + <section id="changing-zoom-in-response-to-user-input"> + <h3><span class="secno">6.4 </span>Changing zoom in response to user input:</h3> + + <pre>function mediaStarted() { + setupRange( videoTrack.source ); +} + +function setupRange(videoDevice) { + var zoomCaps = videoDevice.getRange("zoom"); + // Check to see if the device supports zooming... + if (zoomCaps.min != zoomCaps.max) { + // Set HTML5 range control to min/max values of zoom + var zoomControl = document.querySelector("input[type=range]"); + zoomControl.min = zoomCaps.min; + zoomControl.max = zoomCaps.max; + zoomControl.value = videoDevice.zoom; + zoomControl.onchange = applySettingChanges; + } +} + +function applySettingChanges(e) { + videoTrack.source.set({ zoom: parseFloat(e.target.value)}, true); +} +</pre> + </section> + + <section id="adding-the-local-media-tracks-into-a-new-media-stream"> + <h3><span class="secno">6.5 </span>Adding the local media tracks into a new media stream:</h3> + + <pre>function mediaStarted() { + return new MediaStream( [ videoTrack, audioTrack ]); +} +</pre> + </section> + + <section id="take-a-picture-show-the-picture-in-an-image-tag"> + <h3><span class="secno">6.6 </span>Take a picture, show the picture in an image tag:</h3> + + <pre>function mediaStarted() { + var videoDevice = videoTrack.source; + // Check if this device supports a picture mode... + if (videoDevice.takePicture) { + videoDevice.onpicture = showPicture; + // Turn on flash only for the snapshot...if available + if (videoDevice.fillLightMode != "notavailable") + videoDevice.set({ fillLightMode: "flash"}, true); + else + console.info("Flash not available"); + videoDevice.takePicture(); + } +} + +function showPicture(e) { + var img = document.querySelector("img"); + img.src = URL.createObjectURL(e.data); +} +</pre> + </section> + + <section id="show-a-newly-available-device"> + <h3><span class="secno">6.7 </span>Show a newly available device</h3> + + <div class="note"><div class="note-title"><span>Note</span></div><p>A newly available device occurs whenever an existing device that was being used by + another application (with exclusive access) is relinquished and becomes available for this + application to use. Of course, plugging-in a new device also causes a device to become available. + </p></div> + + <pre>This scenario is not currently possible with this proposal. +</pre> + </section> + + <section id="show-all-available-video-devices"> + <h3><span class="secno">6.8 </span>Show all available video devices:</h3> + + <pre>var totalVideoDevices = VideoStreamSource.getNumDevices(); +var videoTracksList = []; +for (var i = 0; i < totalVideoDevices; i++) { + var mediaStream = new MediaStream( new VideoStreamTrack() ); + // Create a video element and add it to the UI + var videoTag = document.createElement('video'); + videoTag.srcObject = mediaStream; + document.body.appendChild(videoTag); + // Request to have the track connected to a source device (queue these up in the for-loop) + navigator.getUserMedia(mediaStream); +} +</pre> + </section> + </section> + <section id="changes"> + <!--OddPage--><h2><span class="secno">7. </span>Changes</h2> + + <p>This section documents the changes from the prior proposal:</p> + + <ol> + <li>Separated out the Track-type hierarchy from V4 into Track types and Track sources. The sources largely don't use inheritance.</li> + <li>Dropped the device list concept. Instead, simplified the ability to simply find out if there are multiple devices (via the + static <code>getNumDevices</code> method).</li> + <li>Made Video and AudioStreamTracks constructable (an idea for synchronous <code>getUserMedia</code>).</li> + <li><code>PictureDeviceTrack</code> renamed to <code>PictureStreamSource</code> and dropped the semantics of it being a track; it's + now just a special type of device source that shares settings with its video source through inheritence.</li> + <li><code>PictureEvent</code> renamed to the more generic (and re-usable) <code>BlobEvent</code>. (I considered MessageEvent, but + that has too many other unrelated properties, and ProgressEvents didn't have a place to expose the data.) + </li> + <li>Cleaned up the settings that were previously on Video and AudioStreamTrack. Moved them all to the device sources instead. The only + non-settings that remain are AudioStreamTrack's <code>level</code> and VideoStreamTrack's <code>facing</code> values as these + have no corresponding settings to change.</li> + <li>Added a few new settings: <code>gain</code> to AudioStreamSource; <code>mirror</code>, <code>photoWidth</code>, and + <code>photoHeight</code> to VideoStreamSource; <code>bitRate</code> to Audio/VideoStreamRemoteSource. Dropped + <code>dimension</code>. + </li> + <li>The rotation setting was changed to an enumerated type.</li> + </ol> + </section> + + <section id="acknowledgements"> + <!--OddPage--><h2><span class="secno">8. </span>Acknowledgements</h2> + <p>I'd like to specially thank Anant Narayanan of Mozilla for collaborating on the new settings design, and EKR for his 2c. Also, thanks to + Martin Thomson (Microsoft) for his comments and review. + </p></section> + + + + +</body></html>
Received on Friday, 30 November 2012 23:14:54 UTC