- From: Cullen Jennings (fluffy) <fluffy@cisco.com>
- Date: Fri, 16 Nov 2012 22:27:48 +0000
- To: Stefan Hakansson LK <stefan.lk.hakansson@ericsson.com>
- CC: "public-webrtc@w3.org" <public-webrtc@w3.org>
I've read the minutes that say
goran: cullen's draft refers to other drafts
cullen: we will remove that reference.
and tried to figure out which refernce was being discussed. Does someone remember? I'm sure that I was not suggesting removing a reference to the qos draft.
On Nov 15, 2012, at 7:18 AM, Stefan Hakansson LK <stefan.lk.hakansson@ericsson.com> wrote:
> Dom,
>
> thanks!
>
> I have only one comment:
>
> * I think "[NEW] ACTION: Cullen to update draft to remove reference." should not be here, from the context of the discussion I interpret that it is an IETF document that should be updated.
>
> Stefan
>
> On 11/15/2012 11:37 AM, Dominique Hazael-Massieux wrote:
>> Hi,
>>
>> I've just finished compiling and cleaning up the draft minutes of our
>> F2F meeting two weeks ago at TPAC:
>> http://www.w3.org/2012/10/29-webrtc-minutes.html
>> http://www.w3.org/2012/10/30-webrtc-minutes.html
>>
>> The minutes of the Media Capture Task Force meeting on the afternoon of
>> October 30th will be send separately to the media capture task force
>> mailing list.
>>
>> Please send corrections as needed; text-only copy of the minutes
>> embedded below.
>>
>> Dom
>>
>> Web Real-Time Communications Working Group F2F
>>
>> 29 Oct 2012
>>
>> See also: [2]IRC log
>>
>> [2] http://www.w3.org/2012/10/29-webrtc-irc
>>
>> Attendees
>>
>> Present
>> andy_hutton, adambe, hta, stefanh, derf, burn,
>> dan_druta, richt, anant, dom, juberti, jim, matthew,
>> ekr, fluffy, Magnus
>>
>> Regrets
>> Chair
>> Harald, Stefan
>>
>> Scribe
>> adambe, markus, JimB, martin
>>
>> Contents
>>
>> * [3]Topics
>> 1. [4]API functionalities missing in PeerConnection API
>> 2. [5]SDP handling
>> 3. [6]Implementation status
>> 4. [7]General error handling principles
>> 5. [8]Call flows Walk-through
>> 6. [9]State machines
>> * [10]Summary of Action Items
>> __________________________________________________________
>>
>> <inserted> scribenick: adambe
>>
>> hta: the best use of face-to-face time is to be specific
>> ...: people pay more attention campared to mailing lists
>> ... we've tried to make the agenda specific
>> ... we might go with presenters proposal
>> ... there also be discussions
>> ... we should record off-topic topics and move them to the AOB
>> session
>>
>> stefanh: fist session is about API functionality we haven't
>> addressed yet
>> ... this might be the least concrete item on the agenda
>> ... what de we need to do now and what can we postpone
>>
>> API functionalities missing in PeerConnection API
>>
>> [11]Stefan's slides
>>
>> [11] http://www.w3.org/2011/04/webrtc/wiki/images/1/1e/MoreAPIs.pdf
>>
>> stefanh: API topics
>> ... width, hegiht sent over a PeerConnection
>> ... priority
>> ... is media flowing?
>> ... how much bandwith is used
>> ... how much can be used
>> ... how to reject media streams
>> ... echo control
>> ... one thing missing on the slides
>> ... should we support security descriptions
>>
>> (a use case is presented)
>>
>> scribe: one peer sends video with two different resolutions to
>> two other peers
>> ... are people ok with this?
>>
>> fluffy: we should support this
>>
>> burn: you're showing the entrence and the consumer side
>>
>> stefanh: yes
>>
>> hta: there are multiple ways to achive this reuslt
>> ... first question is.. do we need to support this?
>> ... the answer is yes
>> ... then how should we solve it
>>
>> stefanh: this is the most difficult slide in my deck
>> ... application knows about the receiving side
>> ... or the receiving side could tell the sender
>> ... third possibilty is that the app isn't involved
>>
>> matthew: even if the receiver is involved the sender needs to
>> be involved
>>
>> justin: there needs to be some higher limit
>>
>> matthew: there might be a lot of contstraints
>>
>> fluffy: this is one of the reasons we need constraints
>>
>> stefanh: we shouldn't modify the sdp unless we have very good
>> reasons
>>
>> hta: we currently don't use a speakers queue
>>
>> ekr: what do we expect to happen if the receiver asks for a
>> different aspect ratio
>> ... should the PeerConnection rescale
>> ... ?
>>
>> justin: the sender might need to chose the closest thing
>>
>> hta: we want people to separate the case where you really
>> require something compared to the case where you really can't
>> live without something
>>
>> <matthew> side comment: generally we either need to fully
>> specify the SDP the is used to negotiate things like this OR we
>> need contraints. unfortunately when there's contraints, we
>> *also* need to fully specify the SDP that flows between them...
>> unless we choose to have it *only* be an API issue ("direct
>> manipulation" vs. "constraint setting")
>>
>> stefan: is the sending app or sending PeerConnection informe
>>
>> justin: information about the consumer dimensions could be sent
>> as a session description
>>
>> fluffy: the connection to the video tag is interesting
>> ... would it be reasonable to adjust the send size to the
>> consumer size?
>> ... this was a driver for renegotiation callback
>>
>> martin: you don't have the consumer in the initial negotiation
>>
>> fluffy: it's not always true
>>
>> derf: css also impacts this
>>
>> fluffy: a video tag might be specified with a size or not
>>
>> stefanh: there could be several video tags that wants to affect
>> the sender side
>>
>> hta: there's a ultimate fallback video size
>>
>> burn: smart browsers might behave differently
>>
>> matthew: smart browser decisions must be overidden if the
>> developer wants to do something else
>>
>> burn: if we have an API then the browser must take input from
>> the API into account... otherwise the browser can do its magic
>>
>> stefanh: what is the conclusion?
>> ... the receiving side can influence, but we also need an API
>> to let the developer tweak
>>
>> ekr: do we want information from the receiver to bubble back
>> all the way to the camera on the sender side?
>>
>> burn: a browser can use any valid value in a contraint range
>>
>> matthew: if several consumers (local, remote).. who wins?
>>
>> ekr: some constrains are enforced when set and some constrains
>> seems to be enforced continiously
>>
>> burn: we might need a constraints that says "do not change
>> video size"
>>
>> stefanh: recap on proposed conclusion
>> ... receiving side as driver with an API for the app to
>> influence
>>
>> matthew: there's two ways to implement this
>>
>> fluffy: we don't know if we'll use RTCP or SDP to negotiate
>> video size
>>
>> matthew: even if we go with RTCP we need an API to influence
>> how the browser should behave
>>
>> stefanh: let's move on
>> ... next slide
>> ... if the receiver detaches a stream from the consumer.. what
>> should happen,
>> ... ?
>>
>> ekr: what if sender offers something that the receiver doesn't
>> want?
>>
>> stefanh: we'll get to that in a later slide
>>
>> fluffy: we have two operations.. mute (undetectable from the
>> sender side) and to say "don't send me packets"
>>
>> justin: how do you say that I only want a subset of the offered
>> set of streams
>> ... we don't have a good way to express that today
>>
>> stefanh: next slide
>> ... requeseting a certain bandwith
>> ... there's an IETF draft talking about this
>>
>> DanD: I propose a priority param to addStream()
>> ... we can discuss how the priority information should be
>> handeled but we should decide on a mechanism to specify the
>> intent
>>
>> stefanh: addStream() is only called once per stream, it's a
>> problem
>>
>> DanD: we're the right group to specify the priority intent
>>
>> burn: intent means it might not happen
>>
>> <martin> hmm...s/constraint/pref/g ?
>>
>> <ekr> ?
>>
>> <martin> that works too
>>
>> <matthew> there's a difference between "preferences" and
>> "absolute limits"
>>
>> <matthew> i "prefer" 720P, but there's no way i ever want >2560
>> horizontal pixels
>>
>> <burn> no, constraint != intent
>>
>> DanD: at least the browser sholud be able to say that I marked
>> you packets in a certain way
>>
>> <burn> intent == hint, discussions about hints lead to a
>> decision to support both mandatory and optional constraints
>>
>> <matthew> if you just go with direct manipulation then the API
>> only has "settings", and the difference between "constraint"
>> and "preference" is JS code
>>
>> <matthew> but the moment you let the two ends talk to each
>> other via possibly-unmodified SDP blobs then you need places on
>> the monster where it can be poked with a stick
>>
>> Göran: we should describe this case in more detail
>>
>> scribe: can the network trust the diff-serv code points set by
>> the app
>>
>> fluffy: in some environment you can't
>> ... they might provide information
>> ... but a separate request is needed to verify
>>
>> Göran: we need more discussion around use cases
>>
>> fluffy: I think this should be a topic for the AoB section
>>
>> (people agree)
>>
>> hta: new topic - Security for Qos flow labels
>>
>> ekr: there might be two API surfaces needed: tie constraints to
>> media flow, (missed the other one)
>>
>> DanD: I wan't a request that should be handeled by the trusted
>> environment
>>
>> matthew: even if diff-serv cps isn't supported a system may
>> enforce priority anyhow (we need to support this case as well)
>>
>> hta: new topic for AoB - Control of the DSCP interface
>>
>> stefanh: next slide
>> ... other side of the same coin
>> ... feedback on bw..
>>
>> martin: I thin we need generic feedback on constraints set
>> ... there might be existing mechanisms
>> ... for others we might need to add something
>>
>> matthew: e.g., if video has highter priority over slides, the
>> app might want to know if the slides can't be sent at all
>>
>> stefanh: next slide
>> ... sender side pause/resume
>> ... we currently have enable/disable on MediaStreamTrack
>> ... every consumer is affected
>> ... we could have enabel/disable on PeerConnection
>>
>> burn: we don't need that since you could clone media streams
>>
>> hta: we implemented enable and disable as affecting the
>> associasion between a stream and track
>>
>> stefanh: next topic, AGC
>>
>> fluffy: the text on the slides looks good to me
>>
>> <martin> conclusion here was that we don't need to prioritize
>> the agc/noise settings, it's not a preference, it's a setting
>>
>> <martin> it might also be necessary to change this on the fly
>>
>> burn: what happens if you turn AGC on and it's not available
>>
>> fluffy: it should be a on/off setting
>>
>> derf: I hope there's a way to enumerate settings
>>
>> <martin> we need a way to enumerate settings, for sure
>>
>> fluffy: we should require that
>>
>> <martin> agc/noise is a property of a track
>>
>> hta: AGC should be on track level
>>
>> stefanh: next topic
>> ... rejecting streams
>>
>> martin: we need to know if multiple streams are offered
>> ... we need basic methods on the session description
>>
>> derf: example: you offer me 10k streams.. how do I reject
>>
>> DanD: it's not only if your device can do something.. there's
>> also a question if your service can support this
>>
>> martin: we need to know what you get before you can say what
>> you want to accept
>>
>> hta: if we have a mechanisms for the receiver to turn off a
>> stream at any point.. would it be sufficient to turn it off
>> rather than not accepting it?
>>
>> matthew: it might be a resource question
>>
>> ekr: I interpret it as: you set the remote description and you
>> see what streams are offered
>>
>> stefanh: is the sending app informed about a rejected streams
>> ... ?
>>
>> hta: if the use case for rejecting streams is to save resources
>> then the sender needs to know
>>
>> matthew: it's important how quickly can this be done
>>
>> fluffy: can someone send a proposal to the list?
>>
>> (I missed to scribe a lot of stuff while I was talking)
>>
>> stefanh: next thing - AEC
>>
>> fluffy: one param you need is what's going to the speakers
>>
>> <martin> new issue that is related to this, and one that I will
>> bring up later when we talk about this: SDP describes RTP
>> sessions. The one m= section can have multiple tracks. How do
>> we learn of these other than just waiting for arrival of
>> packets on one of those?
>>
>> <hta> adambe said: I sent a proposal to the list with inbound
>> and outbound streams as different objects, which might make
>> this discussion easier; missed having feedback on that.
>>
>> <matthew> another issue that's related to the last slide and
>> not yet decided (and different in different implementations) is
>> what to offer/send before user consent on a sending device
>> occurs. one approach is to say "recvonly" and change to
>> "sendrecv" when the consent happens, the other is to say
>> "sendrecv" and send muted audio + black video until consent is
>> received. recvonly->sendrecv takes the extra round trip to
>> enable.
>>
>> (scribe is missing a lot of stuff here)
>>
>> burn: why do we need API for this?
>>
>> fluffy: you might want to turn it off
>>
>> burn: could this be a browser control?
>>
>> someone: music is a use case where you wan't to turn EC off
>>
>> matthew: broadcast is also a use case where you want to turn
>> AEC off
>>
>> martin: if the browser and user wants differents things, the
>> user setting should win
>>
>> stefanh: final slide - SDES
>> ... wait until RTCWeb has discussed this
>>
>> ekr: yes
>>
>> stefanh: that was easy
>> ... summary slide
>>
>> martin: you could get two different tracks - one with AEC and
>> one without
>>
>> hta: new AoB (MC TF) to topic - Multiple open of camera/mic?
>>
>> martin: AGC, NR should be fairly simple
>>
>> burn: rather than doing this right now we should sumarize what
>> we have decided the last hour and update the summary
>>
>> hta: there are things that needs to be discussed and some that
>> just can be edited in
>>
>> Acoustic Echo Cancellation
>>
>> stefanh: we're 15 minutes before schedule
>>
>> <markus> coffee break
>>
>> <hta> anyone remote actually here?
>>
>> SDP handling
>>
>> [12]Cullen's slides on SDP handling
>>
>> [12]
>> http://www.w3.org/2011/04/webrtc/wiki/images/2/2d/RTCWeb-SDP-API_v6.pdf
>>
>> <markus> getting started, markus is the scribe
>>
>> <markus> fluffy showing slides
>>
>> issues: what SDP do createOffer and createAnswer create and how
>> they can be manipulated before passing them back to the browser
>>
>> matthew:some changes are ok from SDP syntax perspective but not
>> for the browser to act upon
>>
>> goals: clear definition of SDP and error handling rules
>>
>> how are new ICE candidates added to the set SDP
>>
>> Matthew: when I ask which SDP is in use is it the one I have
>> set or can it be something slightly different?
>>
>> constraints: can be used to enable common use cases, but do not
>> solve what can be changed
>>
>> burn:constraints can affect what SDP gets generated in the
>> first place so it does not need to be modified anymore
>>
>> fluffy: the place to define SDP use is in the JSEP draft -
>> latest draft has a start
>>
>> <AndyHutton> remote audio not working
>>
>> matthew:state transitions need to be taken into account, can't
>> call everything multiple times
>>
>> <matthew> specifically, setRemoteDescription("answer") is
>> restricted to only-call-once even though we are saying that
>> enforcement of these transitions is up to the JS
>>
>> what can be changed between create* and setLocal/Remote
>>
>> use cases: remove codec, change bw limit etc.
>>
>> justin: enable/disable bundle is one case
>>
>> adam:rejecting audio, getting video - is that with munging SDP?
>>
>> fluffy:no, if we have explicit API then don't use SDP mungling
>>
>> matthew:have a detailed set of cases
>>
>> fluffy:grouping of cases:
>>
>> fluffy:don't do RTCP mux via settings, not SDP
>>
>> matthew:ptime for codecs
>>
>> matthew:want to fix it
>>
>> matthew:will take my cases to the list
>>
>> [several people agree most things being discussed should be
>> done via other mechanisms than changing SDP]
>>
>> burn: nervous about limiting too much what in SDP can be
>> modified
>>
>> fluffy:changing SDP on the way between browsers can be done
>> flexibly, changing between create* and set* in the same browser
>> more limited
>>
>> justin:don't understand why there is big difference between
>> different manipulation "loops"
>>
>> fluffy:positive list of what can be changed is needed, not the
>> list of what can't be
>>
>> provide a list of changeable things that is *guaranteed* to
>> work
>>
>> matthew: errors need to be specific on what is wrong
>>
>> [argument over what the current spec (JSEP?) says...]
>>
>> burn: many kinds of modifications will be needed before sending
>> SDP out
>>
>> burn:what you send out does not match with what you set with
>> setLocal?
>>
>> <ekr> who was speaking?
>>
>> matthew:jsep-02 has fixed the ice password issue
>>
>> fluffy:need to have the use cases that motivate changes
>>
>> justin:needs coming up in the future anyway...
>>
>> how much and what state does createOffer create? devices with
>> HW codecs?
>>
>> <matthew> jsep-02 fixed the "createOffer is optional" language,
>> though whether or not createOffer creates state is true or not
>> is questionable
>>
>> <ekr> matthew: though, there's clearly an implication of that
>> b/c of the validity window of the offer.
>>
>> <ekr> i.e. during the callback.
>>
>> <matthew> indeed
>>
>> martin:set down the list of things that MUST be possible to
>> change
>>
>> burn:is everything else MUST reject?
>>
>> people have different interpretations...
>>
>> explicit list of things that MUST be changeable, explicit list
>> of things that MUST not be changed and what falls in between
>> the browser need to explicitly error report if it won't support
>> it
>>
>> <martin> there is a list of things that we MUST be able to
>> change; there is a list of things that MUST NOT be able to
>> change, which trigger an error; everything else the browser
>> MUST either accept or reject with an error
>>
>> <matthew> silent failure is incompatible with the assert on the
>> 2nd slide. either the browser fires an error (invalidating the
>> assert) or it is lying to you in order to make that assert.
>>
>> [the above three comments (markus, martin, matthew) try to
>> record the consensus in the meeting]
>>
>> AndyHutton: Configuration/settings? how do they relate to the
>> MUST, MUST NOT, ...
>>
>> fluffy:start with the list of use cases to get the
>> MUST-be-changeable list
>>
>> matthew:listing MUST-items...
>>
>> [cullen taking notes...]
>>
>> adam: we will have APIs for some of these things anyway
>>
>> fluffy:if we have an API to control what createO/A gives, is
>> that not enough, do we still need to change SDP for those
>> "features"
>>
>> justin, matthew: there may be cases where SDP change is still
>> needed
>>
>> fluffy: use cases are still not clear
>>
>> burn:do not worry about MUST-list anymore because anything on
>> the MUST NOT list can still work...
>>
>> still sensing consensus on having three lists: 1. MUST be
>> changeable, 2. MUST NOT be changeable, 3. (default) Browser
>> MUST give an error if it does not support it
>>
>> fluffy:next issue
>>
>> when can two different video flows use the same m-line
>>
>> Proposal: all codec parameters are the same, "content-label" is
>> the same, are in same MediaStream
>>
>> (hta, fluffy, martin debate the details)
>>
>> next: How does createOffer know to offer receiveOnly flow?
>>
>> want to receive video but don't have video camera
>>
>> justin has a proposal, mind to write it down here?
>>
>> matthew: can you put "send" in SDP before getting user consent?
>>
>> or do you first have to use receiveOnly and add sendReceive on
>> a separate O/A
>>
>> ekr: how do we correlate multiple offered video m-lines to the
>> multiple video streams the answerer has
>>
>> next topic: how does createOffer decide to offer a data
>> channel?
>>
>> should OfferToExchangeData constraint be added?
>>
>> matthew: data is a great idea, but SCTP is horrible.
>>
>> fluffy:take this to the IETF
>>
>> tim:SCTP was decided in Feb
>>
>> <ekr> matthew: how do you really feel about SCTP?
>>
>> DanD:data is easy within a single app, but trapezoid between
>> two apps is more difficult
>>
>> martin:issues could come also if the other device running the
>> "same" app has constraints
>>
>> consensus: don't add this constraint
>>
>> next: DTMF
>>
>> will be discussed tomorrow with a proposal
>>
>> next: How long is SDP from createOffer/Answer valid?
>>
>> matthew:90 seconds would be an ultimate timeout
>>
>> use case: the SDP is sent to the server for modification
>>
>> should it be valid beyond the duration of the callback function
>>
>> ice candidates etc. time out in matter of 10s of seconds
>>
>> hta: time-to-live for the session description?
>>
>> matthew:can createOffer be called again after getting the
>> modified SDP back from the server?
>>
>> matthew:proposes that createAnswer is valid only for the
>> duration of callback and no longer
>>
>> consensus: it must be valid at least for the duration of the
>> callback function
>>
>> ACTION: ekr to take follow-up to the list
>>
>> slides about rollback and error handling left for now
>>
>> (now lunch break until 1:30)
>>
>> Implementation status
>>
>> <fluffy> Cullen is scribing
>>
>> HTA is presenting what chrome is doing
>>
>> TURN and opus are scheduled to be in M24 all going well
>>
>> DTMF is waiting on this group
>>
>> Anant has a demo of firefox
>>
>> THe demo allows login, gives a list of users, then you can call
>> one of the other users
>>
>> Have getusermedia, have peerConnection,
>>
>> expectation to not have behind a flag in firefox 19
>>
>> currently in firefox 18 in nightly builds behind a flag
>>
>> have a fairly complete version of the DataChannel
>>
>> showed cool file sharing with drag and drop using DataChannel
>>
>> THey have DTLS-SRTP
>>
>> ICE but no TURN
>>
>> prototype of Identity working
>>
>> Doing desktop first, then working on mobile
>>
>> VP8, opus, and G.711 as codecs
>>
>> HTA: There was a test web even 2 days ago
>>
>> dom: The idea of these is to get lots people to develop and
>> contribute test cases
>>
>> … presentation of theory of testing
>>
>> … Event led to 404 test cases
>>
>> General error handling principles
>>
>> ScribeNick: JimBarnett
>>
>> [13]Anant's slides on error handling
>>
>> [13]
>> http://www.w3.org/2011/04/webrtc/wiki/images/d/d1/TPAC_2012_WebRTC_error.pdf
>>
>> Anant: exceptions vs error callbacks. exceptions when you can
>> detect error without blocking the main thread. Covers only very
>> simple errors. In all other cases, error callback. So as
>> policy, favor error callbacks because they don't block the main
>> thread.
>>
>> Cullen: so lots of functions will do both?
>>
>> Anant: yes
>>
>> EKR: there's no way to get around exceptions occasionally.
>>
>> Tim: what do you do if you pass in something that is not a
>> function as the error callback?
>>
>> Cullen: there are things that you should never see in
>> production code, and those that have to be caught at runtime.
>> The former are programming errors. We're interested in input
>> errors.
>>
>> Anant: what goes in spec is a list of things that must be
>> exceptions and what should be callbacks. This will be the
>> result of consensus of browser implementations.
>>
>> Justin: so programming errors will be exceptions, bad SDP from
>> the other side will be an error callback.
>>
>> Anant: exceptions will be only for development. They shouldn't
>> occur in deployments at runtime. In exceptions should include
>> name and message, which all platforms support We should
>> standardize the names of exceptions, message can be
>> platform-specific, thought should be human readable. stack and
>> linenumber are useful where available. We will inherit from
>> Error object to make SDPErrorObject.
>>
>> ekr: we should create new attribute for SDP error, rather than
>> overloading linenumber.
>>
>> Anant: yes, having different names helps. We should standardize
>> on the ones that need to
>> be machine readable. Human-readable ones can be
>> platform-specific.
>>
>> Dom: W3C policy is to re-use existing names as much as
>> possible.
>>
>> Anant: we will use same object for error callback and
>> exception. So name and message must be present in object passed
>> to error callback. As with exceptions, human-readable
>> properties can be platform-specific. For SDP errors may want to
>> create a new property. Error callbacks should be mandatory. In
>> current spec, they're optional. That makes it easy to make
>> sloppy errors. If they're mandatory, at least you get an
>> exception if you forget to define the error callback. It should
>> never be the case that there's an error and nothing happens.
>>
>> ... In CreateOffer, exceptions: Invalid_callback,
>> invalid_constraints, invalid_state.
>>
>> Dom: these can be webIDL type mismatch errors. We don't need to
>> specify them separately.
>>
>> Anant: we can move invalid state to a callback (for cases where
>> app violates the state machine that we define.) So we won't
>> need to define any exceptions for (most?) functions. setLocal
>> and setRemote can have invalid_sdp as error in callback. Assume
>> that the success callback in set remote isn't called until the
>> description has been fully applied.
>>
>> Cullen: if have setRemote with provisional SDP and then later
>> will apply final SDP. Consider the case where are parsing SDP
>> and acting on it as you go along. Do you have to roll back? It
>> may be hard to do that.
>>
>> ekr: setRemote shouldn't generate callback until it's complete
>> and in correct state.
>>
>> Justin: we need a separate error to signal case that media
>> system is hopelessly busted.
>>
>> Cullen: we need two kinds of errors. Implementation will know
>> whether the situation is
>> hopeless or not.
>>
>> Harald: Invald SDP indicates that you have been able to
>> rollback, so it's not fatal.
>>
>> Cullen: when SDP fails a syntax check and no state has been
>> changed, as opposed to case where what you thought was a camera
>> turns out to be a mouse.
>>
>> ekr: there are cases where the error is reversible, but you're
>> still screwed.
>>
>> Harald: let's put Anant's proposal into the spec, and work out
>> some of these details later.
>>
>> Anant: we can leave the decision on which errors are fatal to
>> the UA. Further discussions of specific error cases on the
>> list.
>>
>> Dan: are error callbacks mandatory?
>>
>> Dom: sloppy programmers can always put in a no-op error
>> callback. On the other hand, if we make them mandatory, we will
>> have to include them in our examples.
>>
>> RESOLUTION: ERROR CALLBACKS ARE MANDATORY. Anant to update
>> spec.
>>
>> So decided that invalid_state will be a callback no exception
>>
>> <dom> [14]Error Types defined in DOM 4
>>
>> [14] http://www.w3.org/TR/dom/#error-types-0
>>
>> Decided that for SDP, will add a sdpLineNumber
>>
>> decided all errors callbacks will not be optional
>>
>> Call flows Walk-through
>>
>> Justin: when you call setLocal that's when UA starts gathering
>> ICE candidates. Get callbacks as each candidate is gathered,
>> another when all are gathered. If call createOffer before any
>> candidates are gathered, it will have just your local address.
>>
>> Justin: if you want to do trickle, have to get offer first and
>> then gather candidates. So must get getOffer callback before
>> all candidates are gathered.
>>
>> Cullen: what things can cause gathering to begin? What
>> indicates how many candidates to gather? How do you know when
>> you're done.
>>
>> Matthew: it has to be a settable parameter whether createOffer
>> callback can fire before you have enough candidates to produce
>> valid SDP. I don't like to get a success callback when the
>> offer is not valid yet.
>>
>> Cullen: one model is you can decide to wait until SDP is
>> complete. Or we can say that you get callback immediately, then
>> get another when you have all the candidates.
>>
>> Harald: must have ICE candidates to be valid.
>>
>> Cullen: but local address suffices for validity.
>>
>> Justin: in Jingle can send offer without any candidates and
>> it's fine.
>>
>> ekr: it won't work with SIP.
>>
>> Adam: we should have matching examples and callflows, including
>> trickle.
>>
>> Stefan: the Chrome implementation seems to work, with and
>> without trickle. So it should be our reference.
>>
>> Justin: we discussed calling createOffer and setLocal early to
>> start allocation, and then add streams later
>>
>> Cullen: we don't have mechanism for declaring dummy streams.
>> What if had new method: gather n candidates.
>>
>> Justin: that's like setLocal with empty m=lines.
>>
>> Cullen: calling app has pretty good idea of how many candidates
>> it will need. At least an upper bound.
>>
>> ekr: two proposals for pre-allocation: - a direct instruction
>> to peerconnection, or some sort of dummy SDP via createOffer
>> and setLocal.
>>
>> Cullen: we have 6 issues to discuss, all at least as big as
>> this.
>>
>> Harald: I'd like to get an overview of all the questions. for
>> this issue, we have two questions: how to know when SDP is
>> sendable, and how to do pre-allocation.
>>
>> Matthew: what happens with resource reservation? What happens
>> when you do all this gathering and find out that the camera
>> isn't available any more?
>>
>> Cullen: next issue. How does receiving side find out what's in
>> the offer, so that it can show the user: Alice is calling and
>> she wants audio and video.
>>
>> Matthew: when you call setRemote, you have no idea how many
>> times onaddstream will be called.
>>
>> Cullen: general model is a callback for each structure, and a
>> final callback saying that you won't get any more. questions:
>> when do you know that you've gotten all mediastreams? When do
>> you know you have all the tracks? When do the callbacks fire
>> with respect to the setRemote callback?
>>
>> ekr: at what point do side effects take effect w.r.t the
>> callback? I think it must be before.
>>
>> Justin: when does peerconnection.remotestreams get populated?
>> When you get a stream, the tracks should be filled in already.
>>
>> Matthew: parse them into the array, and then call
>> processingcomplete.
>>
>> Justin: how about a callback saying the remotestreams value has
>> changed? (It would be an array value, so this callback would be
>> called only once.)
>>
>> Adam: it's more convenient to get the new stream in the
>> callback, rather than having to parse the array to see what's
>> different.
>>
>> Justin: but it's important to know when the changes are
>> complete, rather than getting changes one at a time.
>>
>> State machines
>>
>> [15]Justin's slides
>>
>> [15]
>> http://www.w3.org/2011/04/webrtc/wiki/images/f/fa/WebRTC_States_v2.pdf
>>
>> <inserted> ScribeNick: martin
>>
>> juberti: [presenting on states]
>>
>> martin: does back to gathering happen for moving from "relay"
>> to "all"?
>>
>> juberti: you probably already have all the necessary candidates
>>
>> matthew: is there any way to remove candidates?
>>
>> juberti: ice restart
>>
>> ekr: can we have multiple components with one completed and the
>> other one with no candidates
>>
>> juberti: we are in checking until all components have resolved
>>
>> fluffy: checking encompasses frozen
>>
>> matthew: restart doesn't affect active flows
>>
>> juberti: yes
>>
>> matthew: why not just describe each of the states as they
>> relate to the underlying states
>>
>> fluffy: this is part of what I prepared for the last call on
>> this topic and we rejected that
>>
>> hta: there is no harm in playing audio while video is failing
>> and restarting
>> ... is there any difference betwen connected and completed?
>>
>> juberti: middleboxes might require the updated offer that would
>> be triggered from the completed transition
>>
>> martin: is this application driven or not, can the browser add
>> new candidates and contine?
>>
>> juberti: restart is required by RFC 5245
>>
>> fluffy: the application is going to need to be involved
>>
>> juberti: if you get a new NIC (e.g. WiFi) you might just
>> trickle that
>>
>> hta: transitions to starting are tied to user actions
>>
>> martin: how does the new WiFi candidate fit into this?
>>
>> juberti: that would trigger a transition to connected
>>
>> fluffy: there is an implication that disconnected might
>> transition to failed, in the case where you were connected and
>> you disconnect then something failed
>>
>> juberti: proposes changing name of "starting" to "noo"
>>
>> <hta> IceConnectState -> IceConnectionState
>>
>> juberti: remove onicegatheringchange, and provide just
>> onicecandidate to fire when the gathering state is changed
>>
>> derf: propose to rename iceConnectState to iceConnectionState
>> or something
>>
>> * missed the name conclusion
>>
>> ekr: is this same as the other state machine, just with the
>> states merged?
>>
>> juberti: yes
>>
>> <hta> martin, name conclusion was to use IceConnectedState
>> rather than IceConnectState (I think - my ears are going)
>>
>> matthew: set...(answer) can't be done twice?
>> ... why isn't createOffer and createAnswer shown on the
>> diagram?
>>
>> juberti: once you set...(answer), you may have removed some
>> critical state for the offer, which invalidates some of the
>> answers
>>
>> Summary of Action Items
>>
>> [End of minutes]
>>
>> ***************************************************************
>>
>> [1]W3C
>>
>> [1] http://www.w3.org/
>>
>> Web Real-Time Communications Working Group Teleconference
>>
>> 30 Oct 2012
>>
>> See also: [2]IRC log
>>
>> [2] http://www.w3.org/2012/10/30-webrtc-irc
>>
>> Attendees
>>
>> Present
>> andy_hutton, adambe, hta, stefanh, derf, burn,
>> dan_druta, richt, anant, dom, juberti, jim, matthew,
>> ekr, fluffy, Magnus
>>
>> Regrets
>> Chair
>> Harald, Stefan
>>
>> Scribe
>> DanD, Juberti
>>
>> Contents
>>
>> * [3]Topics
>> 1. [4]Identity
>> 2. [5]API for removing streams
>> 3. [6]DTMF
>> 4. [7]Other business
>> 5. [8]What Triggers Candidate Fetching
>> 6. [9]When is SDP sendable?
>> 7. [10]Security for QoS labels
>> * [11]Summary of Action Items
>> __________________________________________________________
>>
>> <trackbot> Date: 30 October 2012
>>
>> Identity
>>
>> <dom> ScribeNick: DanD
>>
>> ekr presenting
>> [12]http://www.w3.org/2011/04/webrtc/wiki/images/f/f2/Idp-issue
>> s.pdf
>>
>> [12]
>> http://www.w3.org/2011/04/webrtc/wiki/images/f/f2/Idp-issues.pdf
>>
>> ekr: Three proposals on the list
>> ... Proposal 1 - Identity provided by gUM (Thomson)
>> ... Proposal 2: Prompt user after call (Ohlsson)
>> ... Proposal 3 - Site permissions with identity display (EKR)
>> ... Proposals 1 and 2 require the user to explicitly assent to
>> identity
>>
>> fluffy: How does the person answering the call who's calling
>> before answering the call?
>>
>> ekr: Long term consent
>>
>> Mathew: I like to setup the video selection and enable the
>> camera and the identity makes it complicated
>> ... easy to be done with proposal 1
>> ... We need to make sure the browser does not get into the
>> address book without user's consent
>>
>> hta: I'm not happy with proposal 2
>> ... when the green light go on
>> ... I have an origin that is not bound to an identity but later
>> get's bound
>>
>> ekr: Proposal 4 is a hybrid of 1 and 3
>>
>> dom: I don't think we should use a null value for a parameter
>> with a different meaning as a parameter not being set
>>
>> ekr: I'm happy to use a different parameter name for option3
>>
>> justin: There's a difference between site having access to the
>> camera and site recording
>>
>> dom: We're focusing too much on the green light where mobile
>> devices don't have a green light; I want to make sure we don't
>> build security on top of hardware indicators that are not
>> always available
>>
>> martin: the idea is to have an indicator in the chrome
>>
>> justin: there's confusion about what the green light means
>>
>> fluffy: we are adding to the confusion
>>
>> ekr: the green light (the indicator) is supposed to be on once
>> the camera is accessed
>>
>> hta: there is a issue when applications via USB can access the
>> green light
>>
>> martin: if it's on can drain battery without sending any data
>>
>> fluffy: expectation is that when camera goes on light goes on
>>
>> ekr: going over the proposed rule
>>
>> dom: I'm a little bit confused. if the indicator is in the
>> chrome I won't see it if I switch to a different app (on a
>> mobile phone); how does that affect the reliability of granting
>> access to a camera to a peer
>>
>> ekr: we should
>>
>> Mathew: will we be able to check for long term permissions?
>> ... action on ekr to write something up on tainted streams
>>
>> hta: suggest camera access as a topic for the other issues
>> later today
>>
>> API for removing streams
>>
>> hta: presenting what "remove" means
>>
>> fluffy: we should not look at index
>>
>> hta: either we don't remove the streams and you have a fixed
>> index
>>
>> adam: if we reference it by object even if it's removed from
>> the array I can still find it by object reference
>>
>> dom: this is what DOM is doing
>>
>> hta: in this proposal I have two methods to access: one by
>> returning all and iterate another one by name
>>
>> fluffy: why are we handing developers indexes when they can get
>> them by name
>>
>> dom: using index is not a bad thing as long as you don't assume
>> that what it refers to is immutable
>>
>> adam: I don't think you want to have the sequence because
>> developer can store the sequence and can change later
>> ... the msid draft has already a proposal how to name things
>>
>> hta: we need to decide if we want to get rid of the indexes and
>> go with the labels
>>
>> Mathew: legacy devices might not label
>>
>> Adam: we should go with ID's for both streams and tracks
>>
>> hta: we need somebody to write a proposal
>>
>> <martin> there seemed to be general agreement with Adam's
>> suggestion, namely assigning ids to every stream and track and
>> having label used only for human-readable text
>>
>> <dom> ACTION: Adam to update APIs to use mutable arrays of
>> streams in peerconnection with ids [recorded in
>> [13]http://www.w3.org/2012/10/30-webrtc-minutes.html#action01]
>>
>> <trackbot> Created ACTION-59 - Update APIs to use mutable
>> arrays of streams in peerconnection with ids [on Adam Bergkvist
>> - due 2012-11-06].
>>
>> DTMF
>>
>> hta: presenting API requirements
>>
>> hta: proposal to have two new functions on RTCPeerConnection
>>
>> justin: I like the idea of having the callback with tone and
>> duration
>>
>> Martin: the proposal was to have a track (something that looks
>> like a track)
>>
>> hta: what's the factory for that track?
>>
>> Martin: you construct it
>>
>> hta: now you need to the go from the track to the
>> peerConnection
>>
>> Martin: you get the track from getUserMedia, you decorate it
>> for DTMF and attach to the peerConnection
>> ... on the receiving end it's simple
>>
>> hta: If I would be to implement where do I reliably place this?
>>
>> Fluffy: How do you know to negociate for DTMF?
>> ... another one is long tones use cases
>>
>> justin: you can emulate this with the proposal
>>
>> fluffy: I"m fine with getting rid of the long tones
>>
>> Stefanh: I like the proposal. I'd like more control on the
>> outgoing part
>>
>> burn: I think you can have the DTMF track be created by the
>> peerConnection
>>
>> justin: I support hta's proposal
>>
>> hta: I'll modify the proposal to incorporate burn's suggestion
>>
>> Other business
>>
>> hta: reviewing the collected items for discussion
>>
>> What Triggers Candidate Fetching
>>
>> Mathew: Use case is when a web page displays "call agent" . I
>> don't want all the visitors of the page to use my turn servers
>>
>> ekr: This should not be a problem
>>
>> hta: I don't understand why the use case is not satisfied
>>
>> Martin: you want to load the page and show as quickly as
>> possible
>>
>> justin: we have two mechanisms to control the candidates
>> gathering
>> ... you can do that now
>>
>> Martin: this brings us to changing constraints on the fly
>>
>> fluffy: we want to preallocate now were' talking about how to
>> do it in the API design
>>
>> justin: you would know what you need if you call setLocal
>>
>> fluffy: you can optimistically assume two
>>
>> Mathew: you got two use cases: the conference model where you
>> call in to a conference or the public page. We need to support
>> both efficiently
>>
>> fluffy: can we try to make a proposal
>> ... it's less elegant but should work
>>
>> justin: we can do this using setLocal
>>
>> fluffy: I have a different proposal. Have a constraint that
>> defines the preallocated streams
>>
>> ekr: it's fine for me
>>
>> hta: is this sort of creating a pool?
>>
>> fluffy: yes
>>
>> <martin> proposal is to add a new constraint
>> preallocateCandidates, which takes an integer value that
>> defaults to zero. setting this to any other value through the
>> constructor or updateIce triggers the filling of a candidate
>> set pool of that size. final actions are taken on
>> setLocalDescription
>>
>> hta: decision to go with the proposal to create a pool
>>
>> <martin> cullen will take an action to follow up on the last
>> issue (see my last item starting with "proposal")
>>
>> When is SDP sendable?
>>
>> <juberti> Next topic: when is SDP sendable
>>
>> <juberti> When do we know if we have all streams?
>>
>> <martin> my proposal for this was that the success callback
>> would fire, at which time the array^Wcollection would include
>> all the streams
>>
>> cullen: this doesn't let you know whether a stream was added or
>> removed.
>>
>> matthew: that doesn't work in all cases anyway; imagine calling
>> setRemoteDescription twice, the second time before all
>> callbacks have been received.
>>
>> cullen: on all functions that have callbacks, you can't call
>> the function again before the callbacks have dispatched.
>>
>> juberti: this would be setLocal, setRemote, createOffer,
>> createAnswer.
>>
>> ekr: what about getUserMedia?
>>
>> … should be possible to ask for multiple cameras.
>>
>> juberti: I have a proposal to handle the getUserMedia case.
>>
>> adambe: what about onaddtrack events?
>>
>> adambe: do we need onaddtrack when a stream is added?
>>
>> cullen: I read the spec, I think it says any time the remote
>> side adds a track, you need a callback.
>>
>> adambe: there's also an onunmute event on tracks too.
>>
>> adambe: how about only onaddtrack only when a stream is
>> updated, as opposed to added/removed?
>>
>> juberti: I like that proposal.
>>
>> cullen: I don't.
>>
>> martin: Let's address the stream callbacks first.
>>
>> hta: Let's do that.
>>
>> martin: setLocal, setRemote, createOffer, createAnswer should
>> all be non-reentrant.
>>
>> <fluffy> proposal is that that the createOffer / createAnser,
>> setLocal, setRemote, you can not call the same function again
>> if the callback from a previous invocation has not returned
>>
>> martin: during time between setRemote and callback, exceptions
>> should occur on any of these 4 APIs.
>>
>> martin: when you call setRemote, stuff will happen, but the
>> browser will return to stable state multiple times.
>>
>> <fluffy> On set remote, you install all the stuff, then does
>> does callback for onaddstream for each stream, then does
>> callback with null to on add stream, then it call the success
>> callback for set remote
>>
>> dom: do any other APIs do this?
>>
>> anant: in XHR, some things are disallowed while it is running.
>>
>> martin: during state transitions, no other transitions are
>> allowed.
>>
>> anant: you can't call open on the same XHR twice.
>>
>> stefan: might this introduce a timing problem? some browsers
>> are slower than others?
>>
>> dom: I don't think so
>>
>> cullen: I would guess that this is an atomic change, and it
>> takes some time - I would look at some API that has similar
>> needs
>>
>> dom: maybe IndexedDB
>>
>> jimbarnett: call should block instead of throwing exception
>>
>> adambe: if you get onrenegotiationneeded, that could cause a
>> problem
>>
>> cullen: onrenegotiationneeded fires after the success/error
>> callback, if needed.
>>
>> martin: queuing of calls could work OK.
>>
>> cullen: one lock across all 4 APIs.
>>
>> <timpanton> The way most UI APIs deal with this is to say that
>> those functions can only be called on the 'main' thread
>>
>> hta: proposal is: when calling one of these APIs - check args,
>> throw exception if needed, check lock, do work or queue task if
>> needed.
>>
>> <timpanton> if the callback is also invoked on the same thread,
>> this makes the thing queue naturally.
>>
>> hta: nobody argued for throwing an exception instead of
>> queueing.
>>
>> hta: anant to write up the details here.
>>
>> <dom> ACTION: Anant to write up queuing mechanism for
>> set{Remote,Local}Description, create{Offer,Answer} [recorded in
>> [14]http://www.w3.org/2012/10/30-webrtc-minutes.html#action02]
>>
>> <trackbot> Created ACTION-60 - Write up queuing mechanism for
>> set{Remote,Local}Description, create{Offer,Answer} [on Anant
>> Narayanan - due 2012-11-06].
>>
>> <martin> callbacks have to be called on the same thread, there
>> is only one thread
>>
>> juberti: when does onaddstream fire?
>>
>> <timpanton> So how could you get re-entrancy ?
>>
>> hta: onaddstream fires after installation is complete, but
>> before the success callback has been dispatched.
>>
>> <martin> re-entrancy applies only in the sense that the actions
>> associated with the methods take time and so could
>> (conceivably, without these measures) operate in parallel
>>
>> burn: no add stream for failures, naturally
>>
>> <martin> the actions occur on browser-internal threads or "in
>> the network"
>>
>> <timpanton> ok. got it.
>>
>> juberti: when is pc.remoteStreams updated?
>>
>> adambe: before the first onaddstream callback is fired,
>> remoteStreams will be fully up to date with the changes.
>>
>> derf:<same thing>
>>
>> cullen: stream names are confusing.
>>
>> derf: event callbacks need to change to be less confusing.
>>
>> <dom> ACTION: Timothy to write up proposal for new stream event
>> names. [recorded in
>> [15]http://www.w3.org/2012/10/30-webrtc-minutes.html#action05]
>>
>> matthew: if SessionDescription is 3264 SDP, that SDP must
>> always be 3264-compliant.
>>
>> cullen: if you can't do local candidates, we should return an
>> error when trying to write SDP.
>>
>> matthew: will trickle update 3264?
>>
>> cullen: I think we'll need to.
>>
>> matthew: if we're doing trickle, we're going to need to update
>> something.
>>
>> matthew: how do we generate workable SDP when trickling?
>>
>> matthew: Chrome currently generates broken SDP?
>>
>> matthew: How do I get valid SDP in the non-trickle case?
>>
>> cullen: wait for ICE complete callback, then SDP will be fully
>> filled-in.
>>
>> matthew: but what about the initial setLocal? That SDP isn't
>> fully valid.
>>
>> juberti: That is just a subset of the trickle case.
>>
>> matthew: But the SDP still isn't valid.
>>
>> juberti: We are going to solve this with the trickle ICE I-D.
>>
>> … and then the rest of the stuff should fall into place. You
>> can call setLocal with an initial "no candidates" SDP, and then
>> gathering commences.
>>
>> hta: we'll refer this to the IETF rtcweb WG to figure this out,
>> and then we can resume this discussion.
>>
>> Security for QoS labels
>>
>> matthew: Want to see API where packet priorities can be set.
>>
>> hta: culler's proposal does this - gives 3 levels of
>> priorities.
>>
>> cullen: API that provides 3, 4, etc levels
>>
>> matthew: I don't know which one is more important.
>>
>> matthew: data could be above or below media (gaming, higher,
>> file transfer, lower)
>>
>> dand: where would this priority be set from an API perspective
>>
>> stefan: on a track or datachannel.
>>
>> <dom> there was a (currently abandonned) proposal for
>> setPriority on XHR that also had 4 levels FWIW
>> [16]http://ajaxian.com/archives/xmlhttprequest-priority-proposa
>> l
>>
>> [16] http://ajaxian.com/archives/xmlhttprequest-priority-proposal
>>
>> matthew: someone needs to write up a proposal for where these
>> things go.
>>
>> hta: set at initialization, or during the call?
>>
>> matthew: I think it could be initialization.
>>
>> dand: once we get this request, we need a confirmation from the
>> browser that it tried to accomplish this.
>>
>> martin: why?
>>
>> dand: this request can be fulfilled by going to a policy
>> server.
>>
>> matthew: I don't care about marking, I just care about the
>> congestion control prioritization
>>
>> matthew: not all packets will be labeled the same way
>>
>> matthew: either for different streams, or different packets in
>> the same stram
>>
>> hta: need to set priority levels, and have it per track, and
>> have 3 or 4 levels.
>>
>> hta: this will set congestion control/queuing in the browser,
>> and setting of QoS is something for further study
>>
>> matthew: cullen has already written a draft
>>
>> goran: cullen's draft refers to other drafts
>>
>> cullen: we will remove that reference.
>>
>> cullen: we don't want JS to set the diffserv code points, but
>> we do want it to be able to discover them.
>>
>> hta: want mapping from track to 6-tuple?
>>
>> <juberti> ACTION: Cullen to update draft to remove reference.
>> [recorded in
>> [17]http://www.w3.org/2012/10/30-webrtc-minutes.html#action06]
>>
>> <trackbot> Created ACTION-61 - Update draft to remove
>> reference. [on Cullen Jennings - due 2012-11-06].
>>
>> Action. stefanh to propose priority API.
>>
>> <juberti> ACTION: stefanh to propose priority API. [recorded in
>> [18]http://www.w3.org/2012/10/30-webrtc-minutes.html#action07]
>>
>> <trackbot> Created ACTION-62
>>
>> Adjourning for lunch.
>>
>> Summary of Action Items
>>
>> [NEW] ACTION: Adam to update APIs to use mutable arrays of
>> streams in peerconnection with ids [recorded in
>> [19]http://www.w3.org/2012/10/30-webrtc-minutes.html#action01]
>> [NEW] ACTION: Anant to write up queuing mechanism for
>> set{Remote,Local}Description, create{Offer,Answer} [recorded in
>> [20]http://www.w3.org/2012/10/30-webrtc-minutes.html#action02]
>> [NEW] ACTION: Cullen to update draft to remove reference.
>> [recorded in
>> [21]http://www.w3.org/2012/10/30-webrtc-minutes.html#action06]
>> [NEW] ACTION: stefanh to propose priority API. [recorded in
>> [22]http://www.w3.org/2012/10/30-webrtc-minutes.html#action07]
>> [NEW] ACTION: Timothy to write up proposal for new stream event
>> names. [recorded in
>> [23]http://www.w3.org/2012/10/30-webrtc-minutes.html#action05]
>>
>> [End of minutes]
>>
>>
>>
>>
>>
>
>
Received on Friday, 16 November 2012 22:28:20 UTC