See also: IRC log
<trackbot> Date: 09 May 2012
<anant> Zakim: Mozilla is anant
<scribe> Scribe: Josh_Soref
stefanh: Josh_Soref, will you scribe?
Josh_Soref: yes
stefanh: I'd like to add a slot for burn to talk about Constraints
burn: I sent a minor correction to Josh_Soref just now
Josh_Soref: I'll update that and send out new minutes
stefanh: is it ok to approve the minutes with that correction?
RESOLUTION: Minutes from 24 April are approved with burn's correction
stefanh: there were 2 actions from the minutes
... burn to write a proposal
... and myself to contact travis
... i haven't done that yet
burn: i sent an email, about an hour ago
... to the list
... I promised to provide an email with some of the syntax alternatives
... these two, I think represent as much of the consensus
... and interest as I heard
... there were minor variations on these
... I think overall, if we could choose one of the three approaches
... the original, alternative 1, and alternative 2
... but it'll be clear what we're discussing
... the original is copied directly out of the constraints proposal
<dom> Syntax options for constraints structure Dan Burnett
[ Alternative 1 ]
burn: here, i restructured it
... instead of mandatory and optional being the top-level keys
... the top level are video and audio
... with mandatory/optional underneath them
... or for the value of the video key to be "true"
... I had to come up with a value of "optional"
... video is a struture
... with mandatory and optional parameters
... mandatory is an object (dictionary)
... and optional is still a list (array)
... ordered
... the ones that come first have a higher priority
[ Example 2 ]
burn: this uses the syntax shortcut for optional
[ Example 3 ]
burn: this uses the syntax shortcut for mandatory
<Zakim> Josh_Soref, you wanted to note that "optional" would need to be a string
anant: Josh_Soref noted that optional would need to be a string
... we could make them be boolean or dictionary
... and for optional you could not specify the property
burn: that wouldn't be the same
... {audio:optional} means, I'm ok with audio, but i don't need it
... if you say {}, audio means you don't want audio
anant: we'll have to think about this a bit more
... from an implementation perspective
... they could be Boolean, String, or Dictionary
... we might just have String or Dictionary
... that's just a minor detail
... overall, it looks reasonable
adambe: perhaps, instead of making all three stirngs
... could we use an enumeration type?
burn: we could use {video:{mandatatory:{}}}
... Dict + Enum
<Zakim> Josh_Soref, you wanted to note that "enum" is being deprecated
anant: we're trying to move away from enums
... enums become messy things
... navigator.media.ALLOW...
... so strings are better
... we're moving away from them for PeerConnection
burn: any other questions about alternative 1?
[ None ]
[ Alternative 2 ]
burn: same 3 examples
... video and audio have been added to the dictionary
<anant> Josh_Soref: enums are too verbose and require a prefix everytime (like navigator.media) that adds unnecessary characters
burn: we could still do...
... the key difference is that we have {mandatory:{}, optional:{}} at the top
... In the second example of alternative 2, we could modify optional:
... to be an optional value for audio/video as in alternative 1
<dom> (enum is actually a construct in WebIDL to build list of acceptable strings)
adambe: a key difference here
... is that we don't have to prefix the constraints w/ video/audio in Alternative 1
... but they're necessary in Alternative 2
... in A2E1
... is video mandatory in this example, because there's a mandatory Video named constraint
... in Example 1
...
burn: it isn't needed
... if you look at the algorithm
... it walks through to determine if video is requested
... The fact that any video constraint exists under mandatory means that video is required
... if it was only requested in optional, then it was optional
XZ: wouldn't there be a way to say
... "if i get video, i need video with this properties"
burn: i'm not sure if there's a way to optionally request audio with constraints
anant: is there a UC for this?
burn: I can't think of one
... i can't think of a case where you'd say "i'd like audio", and have it optional
anant: we need a bit more thought over there
burn: i could live with that
... i can think of optional constraints
... but not on getting/not getting the stream
<bryan> is the optionality in at least some use cases intended to allow UI to enable the user to decide?
anant: between example 1 and 2, it seems inconsistent
... are the top level keys {mandatory:{}, optional:{}, video:{}, audio:{}}
burn: in Alternative 2, but not Alternative 1
bryan: it seems one UC for optionality is to let the User choose
... the app could work just as well w/ no audio
... you don't know if the user is a Signing user or a Speaking user
anant: you're right
... you could start with Video for Video chat
... and add audio later
burn: this is interesting
... because none of these syntaxes support audio optional, but with constraints
<anant> or, more likely, start audio and "upgrade" to video :)
burn: or video optional but with constraints
... the fact that you have the object
... is essentially saying... you expect to get the stream
<bryan> I'm not sure about the constraints importance in an optional context
burn: however, if they want an optional audio stream
... the reason it's optional is so the user can say "no"
... that doesn't change the desire of the application to request parameters for a stream it receives
Josh_Soref: +1
<bryan> ok, understand better now... +1
adambe: perhaps you could have
burn: so Audio + Video keys
... indicate mandatory / optionality of the stream itself
adambe: then you can have the same constraint set
burn: one disadvantage is that you go back to the longer constraint names
... for a moment, i thought we were about to agree on the approach of alternative one
adambe: sorry for that
burn: i never intended to provide an opportunity for a stream to be optional
... only for constraints to be optional
... i'll probably have to go back and think about this some more
... in order to have it work in the ways we've discussed
... to support optional streams
... i'm not saying it couldn't be done
... but it's non-trivial to address
... so, i'm asking: how important is that?
... i don't think we've ever discussed optional streams
anant: because before we had hints
... and everything was optional
... i think you're fine for deferring this for now
... and maybe we discuss it further on the ML
stefanh: the chairs sent a message last week
... saying we thought it was time to
... BBB
... but maybe we need more thinking time
adambe: we could put it in the draft
... and if we pick alternative 1 or 2
... and we could discuss optional streams later
stefanh: in this meeting, is alternative 1 or 2 preferred
<anant> I prefer alternative 1
burn: without optional streams, i'd prefer alternative 1
... with optional streams, i think alternative 2
anant: i think we could put alternative 1 in now
... and when we rethink things, we could change it
adambe: i think optional streams is written in the notes
... shouldn't it be optional TRACKs?
burn: yes
... you're correct
... i wrote this when we weren't clear
... what will go into the document will very quickly need to change to Track
... if i say Stream, i mean Track
... i guess we could do a quick check
... anant, i see you prefer alternative 1
anant: yep
burn: i'm fine iwth that
Josh_Soref: I think I favor alternative 1
burn: I also favor alternative 1
hta: i think we have consensus to put alternative 1 in the draft
... i will adjust the enums to strings
PROPOSED ACTION burn to write alternative 1 into the draft
stefanh: dom, Actions will go into another tracker?
... for media capture?
dom: yes
<hta> ACTION: burn to write alternative 1 into the draft [recorded in http://www.w3.org/2012/05/09-mediacap-minutes.html#action01]
<trackbot> Created ACTION-1 - Write alternative 1 into the draft [on Daniel Burnett - due 2012-05-16].
stefanh: i sent an email about this
... about whether it would reserve
... initially or when it's used
... there was some discussion on the list
... there's some argument for an application to ask at load time for permission
... but there's a drawback
... "what does a media stream with an enabled track mean?"
... another web/native application could use it
hta: that's not a very clear item
... we don't know if a camera/microphone has exclusive access
... in Chrome today, any number can have the camera open
... so far we haven't implemented constraints
... my assumption, is ideal
... we'd take the
derf: there's of course the possibility that an app other than chrome takes control of the camera
... between request, and attaching to a destination
anant: one thing we're experimenting with
... is having getUserMedia reserve the device
... but not activate it
... if we can't reserve the device, the call fails
... if video was mandatory
... we don't activate the device
... the camera isn't turned on
... and no data is sent
... but when the stream is tied to a canvas/MediaStream
... it is turned on
... we haven't tackled the question of allowing more than one application to have access to the camera
<bryan> Android does not support multiple app access to camera AFAIK
anant: it might have unintended user consequences
<adambe> derf
derf: the thing that concerns me, is that the user could forget the app that asked for permission
anant: the way i planned to address that
<bryan> reservation is an interesting feature but needs to be clear to the user in the permissions UI
anant: is that we'd tell the user that a second thing is asking
... indicating which is using it currently
derf: i'm less concerned with that case
... than with the case of a single page asking
... and not using it for a while
... and the user forgetting
<bryan> the error callback would need to clarify that a reservation error occured
derf: and then the app later turning it on
hta: we're thinking of tray notification
anant: we could show the video active in the tab notice
... and disband when the tab close
adambe: i don't care about copying video buffers around
hta: the distinction is that when you call gUM
... the tray icon appears
... and when you connect to a sink, the light turns on
jesup: not every camera has a light to turn on
hta: right, and some of them are under software control
bryan: if we give the requesting app some indication
... that it failed because it was reserved
... that would let it give some UI to the user
... and the UI for the user on a reservation would need to be clear
anant: i agree
... especially for incoming calls
... if you're on a page, and aren't currently using the camera
... but if you get a call,
... you may want to accept the call and drop the camera
... if the Browser can't get access
... then it'll fail
... but if the Browser has two apps
... then it can switch internally
bryan: if i'm using a code-scanning
... and i get a call
... i wouldn't want the app to fail
anant: i'm describing getting a case which lets the user
... transfer the video from one browser app to another
adambe: if we can't tell the User that the app is currently using it
... in Windows, you always get this frustrating thing
... "the file is in use"
... we have a request, a reservation, and a start reading phase
... talking about reservation
... you don't get the benefits
... the stream can't monitor the underlying source
... if you only reserve
... you have drawbacks
... but you can't monitor the stream
anant: i don't fully understand that
<bryan> oops
<bryan> or me
anant: for practical purposes,
... get ... reservation
... is made
... it's an optimization
... with no sink, we don't need to move frames around
... you can observe it
... but we haven't turned on the camera
adambe: throwing away frames isn't a good thing
... on mobile, that's bad for your battery
... for a long running application
... (chat)
... you want to be able to make a reservation
anant: for a gUM
... the reservation is an internal detail
... for all practical purposes
... if gUM succeeds, you will be able to get frames
... because you have a hold
adambe: it's a bit different than a native app
anant: the incoming call case is different from the chat case
... we want to discourage pages from eagerly requesting
stefanh: i agree with anant
... this seems like the cleaner approach
<Zakim> Josh_Soref, you wanted to note that modern Windows says which app and has a "switch to"
anant: maybe we can write this up and send it to the list
... and if we get consensus, we can draft it into a document
... this may be an authoring guide
... if someone calls eagerly
... and another app asks
... the UA can let the User terminate the early request
adambe: talking about the Stream API
... this would make it more consistent with a network stream
... you have a natural flow all the time
... someone is sending data
... it would make local+remote streams more similar
anant: that seems like a good side-effect
stefanh: we touched on this briefly at the last call
anant: what do we mean by several devices
... is that several mic's + cameras?
stefanh: if you have 2 cameras and want to use both
... do you call gUM multiple times
... or do you get multiple Tracks?
anant: I think we need more UCs in the Scenarios doc
... and even if we have this UC
... how important is this UC?
... we could provide additional Constraint
... changing gUM depends on how important we think this UC is
... if we don't change this, we let the App call gUM multiple times
... where they get 2 MediaStreams, one camera per stream
... in the first case, you get 1 MediaStream with multiple tracks
... it depends on how important the UC is
... and we could just use Constraints
stefanh: you could combine multiple MediaStreams into one
anant: yes, using the MediaStream Constructor
... it depends on input from developers
jesup: how do you determine cameras?
anant: it's user choice
burn: we haven't talked about distinguishing
anant: what is the type of distinction?
burn: user/env front/back
anant: you could have "rotatable cameras"
... that's fuzzy
burn: i agree
... and that's independent of how you specify it
anant: I'm leaning to have multiple gUM calls
... one per camera
... it depends on how important it is
... how frequent we expect this to be
adambe: i think the specification doesn't say anything about this
... and the fact that MediaStreams can contain several streams of each type
... I think we need to specify that the UI should only allow the user to select one of each type
<Zakim> Josh_Soref, you wanted to note that you could have 3 identical cameras that are only distinguishable by the user
burn: at some point we need to make a decision
... if we decide that gUM can only return one Track of each type
... then the way you're describing it is how we should be explicit in the spec
adambe: LocalStreams could only have one of each type
... an empty list is something
... a list of one thing is something
stefanh: i don't know if we should take this to the list
... at this point i'm leaning toward
... I guess we should take this to the list again
stefanh: hta, you proposed this
hta: "this application can use my camera any time"
... if you have stored permissions
... you need a UI for managing them
... Chrome for picking which camera to use
... is only invoked at checking permissions
... then if you've given an app permission
... how does the app get back to the Camera selection dialog?
jesup: I think you need to be able to have a way to select the selection dialog regardless of this
... wanting to have a way to flip between front and back on a mobile device
... that problem has to be addressed anyway
... possibly another gUM call
... our thinking on this has been
... having ongoing permissions tied to app installation
hta: yes, that's one way of managing
... if you have an app permission mechanism
... and everything else lives with temporary permissions
jesup: that's our current way of thinking
hta: for getting the dialog when you wouldn't normally need to have them
... is to have a constraint "really ask the user"
burn: constraints for asking permissions
... i don't understand
hta: something in the constraints dictionary that says
... "ask user to pick device"
... ask-user-yes
adambe: app has permissions
... but wants to prompt?
hta: app has permission for front
... but wants the back camera
adambe: i guess the browser database needs to specify which camera the app has been authorized to use
... then if you use gUM
... you can get a prompt for the camera you didn't previously authorize
hta: so a db w/ each camera is its own permission domain?
adambe: it doesn't sound nice
... but perhaps
... i wouldn't be comfortable with the application being able to freely swich
... if i gave permission for one
... for it to switch from one, to the next, to the next
jesup: if you gave the app permission to all of your cameras?
adambe: i guess so, it'd be a different kind of permission
jesup: the only ones you'd be giving this to
... are ones you trust at a desktop/plugin install
... which gives full access
... i don't know if there's something to be gained for such fine grained control
adambe: i agree with you
... it's only risky applications on the web
jesup: for stored applications
... because of the trust you're giving them
... i think All or None is enough
... it simplifies the UI
adambe: that kind of application wouldn't use this UI at all
... it could put up a self view of all cameras
... and say "which do you want to use"
... in gUM, it's selection + permission
jesup: correct, someone with that kind of app might want to do it themselves
<Zakim> Josh_Soref, you wanted to cry
Josh_Soref: ...
... there are cases where a user wants to use a "Camera" app
... but doesn't realize it's going to randomly snoop on the other camera
... There's also the fact that the App wants to use the system picker
... because it doesn't want to build the UI itself
... or because it wants to have the standard system look and feel for the camera picker
jesup: I agree with the second case
... I don't think there are many env- only camera applications
... that would never want to access the other camera
... e.g. instagram would probably want to be able to use either camera
... there might be a UC for a native UI to toggle cameras
adambe: do you think there might be a UC for
... a toggle button
jesup: I think for a dedicated app, like skype
... an in App button is pretty common
adambe: in the other case
... toggling from an external ui
... wouldn't it be problematic
... if you send a stream with a resolution/framerate
... and you toggle
... and the JS application gets a different frame-rate/resolution
... do you think that should be invisible for the app
jesup: i would have no problem with the application getting an event notification
... i think the system needs to be able to deal with those changes anyway
... and applications need to be able to deal with them anyway
... if need be, changing resolutions on streams
... i don't see a problem there
... there could be ways where it happens now
... if the user supplies a Video file instead of a Camera
... the file may have resolution/framerate changes in it
... you have no ability to change it, you have to adapt to it
adambe: it's quite important to be able to mute the camera from the browser ui
jesup: absolutely, critical
... because you can't trust the app to do it
adambe: that would also be a stream change
jesup: the camera may stop sending video
... change its frame rate
... you can't count on the camera to keep a constant frame-rate
<Zakim> Josh_Soref, you wanted to note that users don't read permissions dialogs - ever
hta: i think we have gotten a good handling of the issues
... i don't see a concrete suggestion about language
... anyone else want to speak on this topic?
... stefanh: any other topics?
stefanh: there is quite a long list of open items
... i think we need to start addressing them
... any input would be valuable
<stefanh> http://www.w3.org/wiki/Media_Capture#Open_Items
hta: the WebRTC meeting is coming up
... June 11 in Stockholm
... followed by 2 days of RTCWeb meetings
<dom> WebRTC Meeting June 11
Josh_Soref: there was some mention of MediaStream in the HTML WG F2F last week, I hope to send minutes for it later this week
<hta> ACTION: anant to describe that getUserMedia reserves the devices, causing the "camera to appear busy" [recorded in http://www.w3.org/2012/05/09-mediacap-minutes.html#action02]
<trackbot> Created ACTION-2 - Describe that getUserMedia reserves the devices, causing the "camera to appear busy" [on Anant Narayanan - due 2012-05-16].
<hta> ACTION: anant to add a UI guidelines about "camera is busy because of X" when another request is made [recorded in http://www.w3.org/2012/05/09-mediacap-minutes.html#action03]
<trackbot> Created ACTION-3 - Add a UI guidelines about "camera is busy because of X" when another request is made [on Anant Narayanan - due 2012-05-16].
Josh_Soref: if app 1 calls getUserMedia
... and gets a camera
... and app 2 calls getUserMedia
... then the user can be shown a dialog with both cameras
... the in use one could appear in use but with a preview
... if the user selects that in use camera
... then the UA can indicate the other Web App that is using it
... allowing the user to steal it
anant: For Desktop cases,
... where the Camera is in use by another app running as the same user on the system
... we could steal it
... but we couldn't if it runs as another user (or more privileged user)
stefanh: I'd like to thanks Josh_Soref for scribing every meeting
<anant> Josh_Soref++
stefanh: really grateful
[ Adjourned ]
trackbot, end meeting