- From: Francois Daoust <fd@w3.org>
- Date: Tue, 08 Nov 2011 16:39:08 +0100
- To: "public-webrtc@w3.org" <public-webrtc@w3.org>
Hi again,
The minutes of the second day of last week's F2F meeting are available at:
http://www.w3.org/2011/11/01-webrtc-minutes.html
... and copied as raw text below.
See previous disclaimer for possible inaccurate or incomplete exchanges. I'll send a summary of the meeting later on.
Thanks,
Francois.
-----
WebRTC WG F2F Santa Clara - Day 2/2
01 Nov 2011
[2]Agenda
[2] http://www.w3.org/2011/04/webrtc/wiki/October_31_-_November_1_2011
See also: [3]IRC log
[3] http://www.w3.org/2011/11/01-webrtc-irc
Attendees
Present - group participants
Harald_Alvestrand, Adam_Bergkvist, Dan_Burnett,
Francois_Daoust, Dan_Druta, Christophe_Eyrignoux,
Narm_Gadiraju, Vidhya_Gholkar, Stefan_Hakansson,
Cullen_Jennings, Kangchan_Lee, Wonsuk_Lee, Kepeng_Li,
Gang_Liang, Mahalingam_Mani, Anant_Narayanan, Youngsun_Ryu,
Youngwan_So, Timothy_Terriberry, Rich_Tibbett, Justin_Uberti,
Milan_Young
Present - observers
Mauro_Cabuto, Suresh_Chitturi, Mohammed_Dadas, Steve_Dennett,
Shunan_Fan, David_Yushin_Kim, John_S_Lee, Ileana_Leuca,
Alistair_MacDonald, Chris_Rogers, Doug_Schepers, Junya_Yamada
(a few other observers attended the meeting)
Chair
Harald_Alvestrand, Stefan_Hakansson
Scribe
francois, burn, Milan, fluffy, anant, derf
Contents
* [4]Topics
1. [5]MediaStream (continued)
2. [6]PeerConnection (Cullen)
3. [7]Status of Audio WG
4. [8]Implementation status
5. [9]Incoming notifications
6. [10]Any other business?
* [11]Summary of Action Items
See also: [12]Minutes of day 1/2
_________________________________________________________
[12] http://www.w3.org/2011/10/31-webrtc-minutes.html
MediaStream (continued)
[resuming discussion from yesterday's meeting]
adam: some confusion on whether 5.1 audio should be 6 tracks, or 6
parallel channels of a single track. I tend to prefer the second
approach, but suggest to move on to the mailing-list.
cullen: it sounds good to come up with a decision
hta: 2 alternatives: 1) a track is whatever has one sensible
encoding and 2) a track is whatever cannot be subdivided
adam: cannot be subdivided in JavaScript.
hta: in the case of 5.1, there are encodings that will take 5.1 as
one entity that JavaScript will have trouble decoding.
cullen draws a diagram with stereo channels on white board ([13]Fig
1)]
Media channels (e.g. stereo channels) may appear in a track.
Multiple tracks may take part in a Stream. Streams may be
synchronized. The question is: is WebRTC MediaTrack at the 'track'
level or 'channel' level?
Fig 1 - Stream, track, and channel
anant: question is does JavaScript need access to the channels?
cullen: it has to be addressable e.g. for one video to go to one
monitor and the other to the second one.
adam: why do they need to be addressable?
... they're two different things
cullen: no, they're not.
... [taking example of audio with multiple channels]
... I was assuming they need to be addressable.
anant: only problem is that track means something else in other
contexts.
burn: if we only need to come up with a different name for track,
not a big deal. It makes more sense to focus on the definition of
the term than on the term itself.
anant: I think JavaScript needs to have access to different channels
e.g. to decode things itself.
hta: provided it should be able to do that
[more discussion on audio tracks]
cullen: you need one more name, English won't be encoded with
Russian.
[Fig 1 completed]
burn: 3 levels now. Is there a possibility that we'll end up with
another, and another?
adam: I don't think we need to go any deeper than channel.
cullen: some possibility to group tracks in pseudo-tracks in some
contexts
burn: if we agree on the top level, and the bottom level.
hta: for the moment, we have stream, track and channel.
adam: question is whether channel has to be a JavaScript object.
hta: to say that the right channel should go to the left speaker,
that's useful.
cullen: if you have polycom setup, you can end up with 4 different
channels, that may be encoded together.
stefan: we have tracks mainly for legacy with the media element.
cullen: the grouping is going to go to the other side through SDP, I
think.
tim: that square box is a single SSRC. MediaStream is a single
CNAME.
cullen: ignoring these points for the time being.
stefan: I think synchronization should be on an endpoint basis
hta: you can't do that.
stefan: that's my view, we'll discuss that later on.
cullen: we're all confused so need clarification.
anant: for video, if you have the context of channels, what would
that be useful for?
cullen: stereo video.
adam: the channels are dependent on each other.
anant: I don't understand why channels have to be exposed in the
context of tracks.
[discussion on relationship with ISO base file format]
<Travis_MSFT> Can a "track" be a text caption too?
francois: wondering about mapping to media element. MediaStreamTrack
(or channel) will have to be mapped to the track in an audio/video
element for rendering in the end. Needs to be clarified.
hta: seems we need some audio expertise. Perhaps discuss during
joint session with Audio WG.
adam: moving on. We removed parent-child relationship within a
stream. Good opportunity to make MediaStreamTrackList immutable.
... I don't have a strong opinion, but it simplifies things.
anant: if you clone a MediaStream and disable a track on cloned
object, it will disable, not remove the track.
... The right way to fix this is not to make MediaStreamTrack list
immutable.
... If someone decides to disable the parent track, the child should
receive an event that the track is disabled. Same as if the track
was received from a remote peer.
adam draws [14]Fig 2 on the paperboard]
Streams are independent without parent-child relationship. A child
stream's enabled status depends on that of its parent otherwise.
Fig 2 - Cloned streams with and without parent-child relationship
[exchanges on parent-child relationship, enable-disable events]
tim: the worst situation is starting up the call, adding a video
stream, then an audio stream, then something else.
cullen: then unplug a camera.
hta: it's disabling tracks. addStream is the one thing you cannot do
with enabling/disabling.
anant: we need some kind of grouping for synchronization.
adam: difference between mute and disabled. If I disable on my side,
track is muted on your side.
... Should the track be removed from your side?
... I might re-enable it later on.
anant: should we have the concept of who owns the MediaStream?
... I think the WhatWG thought about it, leading to the concept of
local MediaStream.
... You are in control of local MediaStreams, not on remote
MediaStreams.
hta: it makes a lot of sense to disable things received from a
remote peer.
... When you're changing from disabled to enabled, you're telling
the other side that you'll do something with the data if you receive
it.
<juberti> francois: would you mind dialing the polycom when you get
a chance?
anant: It's possible for a track to be enabled without receiving any
data. That makes sense.
<juberti> i can hear the room well, thanks
anant: no real difference between mute and disabled.
cullen: I was assuming you were sending data in the case of mute,
and not in the case of disabled.
anant: that's just a waste of bandwidth.
<juberti> there are good reasons to send zeroes.
anant: any sensible scenario?
cullen: no idea.
<juberti> some gateways will disconnect if data is not provided.
richt: perhaps if re-negotiation is noe possible.
ileana: there are cases when you press mute and send music.
anant: does that level of semantic need to be exposed to JS?
... does it have to be a change of state in the object?
... Use cases?
<juberti> Call waiting is the main use case for "hold".
adam: we need to have some sense of "mute" for streams you don't
have control over. Not sure about the best word.
anant: I would assume that you can remove tracks from a stream.
<juberti> Local muting (sending zeroes) has obvious use cases.
[discussion on adding/removing tracks in SDP]
adam: for removing tracks, disconnecting a camera is a good use
case.
... Symmetry for adding tracks sounds important.
francois: same discussion in HTML WG through Web and TV IG concerns
on adding/removing, enabling/disabling, tracks to the video element.
It seems important to end up with the same interface for both
contexts.
stefan: in the end, the first bullet point is removed here
('immutable MediaStream objects')
adam: and we'll make updates for the remaining points.
... moving on to getUserMedia
... only difference between local and not is you can stop on
localMediaStream.
anant: why do you need to have a stop method?
tim: if I cloned a track in different streams, you have to stop each
track individually. With stop, that's taken care of.
hta: now you're introducing a dependency that I thought we had
agreed to drop.
adam: I think "stop" on a local MediaStream shouldn't affect other
MediaStreams.
... you need "stop" to say you're not going to use the object
anymore.
anant: I don't understand why it's not possible with remote media
streams.
hta: I guess for the time being, the problem is that "stop" is not
clearly defined.
adam: moving on to recording.
... MediaStream.record(). You get the data via an async callback.
cullen: what format does the recorded data follows?
adam: that's an interesting question.
... when that got specified, I think there was this romantic idea
about a simple unique format.
anant: getRecordedData means stop the recording and give me the
data?
adam: not that much.
cullen: it's awful.
hta: yes, I suggested to delete it while we find something working.
burn: what does it have to do with WebRTC, sending data from one
peer to another?
anant: we need to define getUserMedia, so that's one use case we
need to tackle.
cullen: we should start from scratch and design that thing. Need to
specify format, resolution, etc.
... Why don't we start by sending some good list of requirements to
this working group?
hta: At a minimum, a recorder that returns blobs has to specify what
the format of the blog is.
anant: Additionally, we could use hints here, so that caller
suggests what he wants.
hta: given the reality of memory, we need chunks, so that running a
recorder during two hours doesn't mean it will try to buffer
everything in memory.
anant: I don't get the use case for getting the recorded data back.
It should go to a file.
cullen: somebody else might want to access the data to do other
things such as barcode recognition.
anant: not the recording API, that's the MediaStream API.
hta: you might want to move backward, e.g. in the hockey game to
move back to an action.
... My suggestion is to nuke it from the draft for the time being.
Once we have MediaStream, we can figure out how to record.
richt: you can call record multiple times. Does that reset?
adam: you get the delta from last time
tim: possibly with some overlap depending on the codec
burn: maybe check with Audio WG, because they are also doing
processing with audio streams. Recording is just one example of what
you can do with a media (audio) stream.
richt: we're going to record to formats we can playback in our video
implementation today, and I guess other browsers will probably do
the same, so no common format.
[summary of the discussion: scrap the part on recording for the time
being, trying to gather requirements for this, to be addressed later
on]
adam: on to "cloning".
... With the MediaStream constructor, you can clone and create new
MediaStream objects without prompting user.
... If you have one MediaStream object, you can only control it in
one way. Muting mutes tracks everywhere.
... With cloning, you can mute tracks individually. Also used for
self-view ("hair check screen")
... Pretty nice thing.
... Cloning is creating a new MediaStream from existing tracks)
hta: if you have one MediaStream and another one that you want to
send over the same PeerConnection, you can create a MediaStream that
includes both sets of tracks.
adam: composition: same approach. Used to pick tracks from different
MediaStreams.
... e.g. to record local and remote audio tracks in a conversation.
... Discussion about this: obvious question is synchronization when
you combine tracks from different MediaStream. Combining local voice
track with remote voice track, how do you do that?
cullen: if we have grouping for sync, can't we reuse it? If same
group, sync'ed, if not, no sync.
... We're synchronizing at global times, so no big deal if local and
remote streams.
hta: works if different clocks?
cullen: that's what RTP does. Works as well as in RTP.
... I think it's taken care of by whatever mechanism we have for
synchronization.
... another thing on cloning and authorization. If authorization on
identity, we might need to re-authorize on the clone if destination
is not the same.
... May not bring changes to the API.
hta: that is one of the reasons why authorization on destination is
problematic.
... If it's irritating, people will work around it.
tim: If I want to authorize sending to Bob, I don't want JS to be
able to clone.
hta: what you care about is that it doesn't send it to Mary.
anant: the data can go anywhere, there's nothing preventing that.
... getUserMedia allows the domain to access camera/microphone, not
tied up to a particular destination.
hta: I think ekr's security draft goes to some details on what we
can do and limitations. Identity providers can mint names that look
close to the names the user might expect to see.
adam: we need to solve authorization without cloning anyway, because
of addStream.
hta: if you want to do that, you have to link your implementation of
PeerConnection with the list of authorized destinations.
[more exchanges on destinations]
DanD: depending on where you're attaching to, you can go on
different levels of authorizations. If I have a level of trust, then
I can say I'm ok with giving full permissions.
anant: problem is where you do permissions. PeerConnection level?
cullen: implementation may check that crypto identity matches the
one on authorized lists of identities that can access the camera.
Travis_MSFT: we still need authorization for getUserMedia.
richt: this authorization stuff on PeerConnection is quite similar
on submitting a form where you don't know where the submitted form
is going.
... Not convinced about this destination authorization.
<burn> Scribe: burn
adambe: (missed this)
anant: how do you correlate different media streams
hta: with only local media it's easy, but mixer has media from
multiple sources and they will be unsynced
... a single media stream, everything in it is by definition synced.
but across media streams no guarantee of sync
fluffy: makes sense. could get cname from them, but doesn't
necessarily tell you how they sync
<Milan> scribe: Milan
PeerConnection (Cullen)
Slides: [15]WebRTC PeerConnection
[15] http://www.w3.org/2011/04/webrtc/wiki/images/e/ea/PeerConnection_v3.pptx
fluffy: Split work into interface types
... current spec isn't very extensible
... wrt sdp
... propose wrapping sdp in json object
... need to differentiate offer from answer
... so add this to sdp JSON representation
fluffy: offer answer pair contribute bits to caller session id and
callee session id
... answer always have an implicit offer
... offers can result in more than one answer
... each media addition results in a new offer/answer pair
... replaces previous
... increment sequence id to note replacement without setting up
entirely new call
Anant: If sdp doesn't result in new call, does it require user
consent?
Fluffy: Yes, depends upon the class of change. EG new video camera?
... require and OK to the offer answer
... can only have one offer outstanding at a time
... need to know when can re-offer (timeout)
Stefan: Also need to know when offer accepted for rendering RTP
streams
Anant: Why sequence ids?
Fluffy: need to suppress duplicates
Anant: Isn't websockets a reliable protocol?
Harold: Not really
... because http can close connection at any time
Stefan: If A makes new offer, when would B side be able to make new
offer?
Fluffy: Depends on order of delivery
... so OK required
Suresh: Are these formats in scope for peer discussion?
Fluffy: Yes, APIs will define how to process these messages. Work
performed by browser vendors.
... ICE is slow
... it is a strategy for finding optimal paths (uses pinholes)
... as such can't send packets too fast
... start processing as early as possible. But need to wait for user
authorization on IP address
... amount of time deciding to accept call should mask ICE delay
... so need a tentative answer followed by final answer which was
confirmed by user
... this is implemented by adding another flag to sdp/json object.
"morecoming" (see [16]Fig 3)
Answerer may partially answer an offer with a first ROAP response,
provided it sets the flag 'morecoming' in the first answer.
Fig 3 - "More coming" temp answer flow in ROAP
DanD: Why not implement as a state machine?
Anant: Why can't OK contain SDP?
DanD: How much time does it buy you?
Fluffy: Time is function of user input
... and also if IP address is protected information
DanD: Maybe browser should delay response
Fluffy: This flag is optional
... this should be part of the hint structure
Harold: User interaction must be between offer/answer regardless
RichT: Maybe we could use the 180 in place
Fluffy: Might confuse forking issue and early media
If you get back calllee sessionid that is different, how do you know
that should result in same or replacement?
Fluffy: This is same question as to allow forking
Harold: If you get back calllee sessionid that is different, how do
you know that should result in same or replacement?
Fluffy: Don't like passing STUN and TURN credentials
... would rather get an offical API from IEFT
Eric: Proposed format is not strongly typed
... makes extensibility difficult
harold: For example whether to provide IP address quickly
... Action add hint structure to peer connection constructor
Fluffy: Turn servers require credentials
... so need to pass them around in JS
Anant: username/password@domain
Richt: Should this be mandatory parameter, or can browser
substitute?
Anant: Browser should provide default
Fluffly: Common to force all media traffic through single turn
server
... option for browser to be configured for turn server
... ice will choose best
... Also need application to provide prefered turn servers
Harold: Google does http transaction before setting up turn service
... need to be able to stop serving dead users
... so need a provisioning step
... so this scheme is consistent
... must pass password at start of peer connection
Anant: If JS object, then it's in developer control
Fluffy: Lable useage is broken
Harold: Thought was that each label was a new RTP session?
... Have need for sending information peer-to-peer for nominating
tracks as being in focus
Stefan: Metadata much also be passed as well as lable
... for example the meaning of the microphone channel
... need to interop with legacy equiptment
DanB: How many thinkgs do we want to label?
Fluffy: I want identifiers on the channels
Tim: Should the list of channels in a given track be mutable?
<juberti> label usage is broken
Harold: Track is defined by its properties
DanB: Haven't decided on the definition of a track
Fluffy: ROAP deals with glare
... SIP works on waits
... but can take too much time
... better solution based on random timeouts where larger wins
Fluffy: need DTMF for legacy systems
Anant: Should live in a seperate track in same stream
Harold: Use case for receiving DTMF?
Fluffy: Yes
Harold: What about analog DTMF?
Fluffy: That is translated by the gateway
Milan: What about SIT tones?
Fluffy: Seems reasonable if easy
RichT: Could this fit into proposal where inject additional media
into stream?
Fluffy: Injecting crappy sound is cool, but not a general solution
for DTMF
Anant: Way to handle that is to define alternate track types
Francios: How do we integrate ROAP into current API?
Harold: Easy. Just replace sendSDP with sendROAP
Status of Audio WG
Have Al & Chris from Audio WG with us
Al: Few proposal roc's work and Web Audio API
Chris: Web Audio API deals with arbitrary media graphs
… we may be interested in basic convolution, equalization, analysis
etc. other effects
Chris: Robert's work provided other cases including remote peer to
peer
… Chris proposed some code examples of how to integrate with rtcweb
spec at
[17]https://dvcs.w3.org/hg/audio/raw-file/tip/webaudio/specification
.html
[17] https://dvcs.w3.org/hg/audio/raw-file/tip/webaudio/specification.html
… examples at
[18]https://dvcs.w3.org/hg/audio/raw-file/tip/webaudio/webrtc-integr
ation.html
[18] https://dvcs.w3.org/hg/audio/raw-file/tip/webaudio/webrtc-integration.html
Chris: Audio API is implemented but connection with RTCWeb not done
yet but confident we could implement this
Anant: What does createMediaStreamSource do ?
Chris: It creates a media stream that can be added to the peer (more
to answer I missed)
... stream can be more or one channel but defaults to two
Harald: If you have a bunch of microphones in room - are these 3
channels in a stream or ...?
Chris: Can process each stream through a separate filter.
anant: Did it have any idea this is 4 channels
Chris: probably need something more like createMediaTrack
anant: We see it as we have streams, that have a set of tracks, that
have one or more channels
Chris: The connections between the nodes are sort of like a webrtc
track. The connection between the audioNodes is a bit like a RTCweb
track
... have direct way ways to get the channels / tracks and combine
into a single stream
... Makes more sense to do these operations on tracks than on
streams
anant: Do you have code showing operating on channels instead of
tracks
Chris: See
[19]https://dvcs.w3.org/hg/audio/raw-file/tip/webaudio/specification
.html#AudioChannelSplitter-section
... The splitter and merger can be used to extract the channels from
a track or recombine
[19] https://dvcs.w3.org/hg/audio/raw-file/tip/webaudio/specification.html#AudioChannelSplitter-section
anant: Can the channels be represented by JS objects
Chris: no - (liberally interpreted by scribe) more the processing is
represented by objects and the connections between are more what
RTCWeb thinks of as tracks / channels. The channels are accessed by
index number
... can combine with other media, mix, and send to far side
... Dynamo (sp?) did apps that send information over werbsockets
that allows people to create music locally using this
anant: how do you deal with codecs / representation
Chris: linear PCM with floating point using typed array
... want clean integration with graphics and webrtc
... most expensive operation is convolution to simulate sound of a
concert hall
… this is implemented multi threaded
… multiple implementations of different FFTs and such
anant: Processing in JS?
Chris: much of the processing done in native code
… Can create node with custom processing with JS
Cullen: have you looked at echo cancelation
Chris: not yet but have code from the GIPS stuff
... can use audio tag or video tag and connect with this
Stefan: Have 4 requirements. F13 to F15 and F17
Chris: F13 can do with placement node
… F14 classes RealTimeAudeiNode - probably better to make new
AudioLevel node that just did this simple function. This is easy and
we plan to do it.
… F15 Can use the AudioGain node
… F17 yes can be deal with Mixer
… that happens automatically when you connect multiple things into
an input
fluffy: Question about mixing spoken audio to bit 3 of top speakers
Chris: could do this
Harald: Can you create events in JS from detecting things in audio
Chris: could build something to add an event to a level filter or
something
... no gate, but have compression that is similar
Doug: How does this compare to ROC proposal with respect to library
?
Chirs: ROC proposal has all audio processing in JS
Chris: there is a JS node that allows you to create processing
effects in just JS, you can associate an event this way by writing
some code to detect a level and trigger an event
Chris: simple common audio , EQ, dynamics etc are very common
… library provides nodes for these
… for specialized, have a customer JS node
Harald: concern with JS is worry about latency
Chris: agree and very concerned about latency in games
… for something like guitar to midi to audio, need very low latency
Doug: with voice, don't want that lag effect where you have talk
over
Chirs: JS would add to the latency
… for something things such as trigger on signal level, latency
would not matter as much
… could have latency in the 100 to 150 ms range
fluffy: what sort of latency would we typically see on some EQ
Chirs: working on 3.3 ms frames of latency
anant: which API is likely to be accepted?
Doug: plan to publish both as first public drafts (a very mature one
and less matter one)
… It is shipping in nightly of webkit
… If Doug had to bet, since we have mature spec, something that
works well, have specs, have implementation, Doug would guess lots
of interested in web audio API
… sounds like existing spec can easily be made to meet needs of
RTCWeb
Tim: roc code runs on worker thread so latency is far less than 100
ms in most cases
Eric: In resource constrained environment, better to have processing
done in native where it can take advantage of hardware resources of
the host
Doug: the difference at an overly simplified level is do you do
mostly JS with some extensions for native speedup or do you do
mostly native with an escape hatch to do things in JS
Chris: procession in a web work still have dangers - such as garbage
collection stalls
… garbage collection stalls can be in excess of 100 ms. Depends on
the JS engine
Harald: Is the web work stable and widely implemented?
MikeSmith: in all major browsers and has not changed in long time
… does not mean Hixie won't make changes.
Stefan: should we bring over some requirements to the WG
Doug: Send us the minutes
anant: Need to make sure the representations of the audio can be the
same and we can make this work together (media steams in one need to
match up with the other)
Chris: have similar problem linking to audio tag
Doug: Good to have doc how how to bridge gaps between the two and
make is easy for someone authoring to these two things
Chris: hopefully have concrete implementation where we can link this
together with peer connection over upcoming months
... have working code in webkit. Gnome GTK port has working code.
Can share code with Mozilla to help move the code in
… Bindings of IDL and such will be some tedious work but tractable
problem
Chris: the DSP code is C++ code
… on mac can use accelerate framework
… Convolution use intel mpl, have ffpt gmpl, have ffmepg fft, call
apples.
Harald: Have fair picture of where we are. Thank you. When getting
specs settled down, can use this. Hope things finished on right
schedule
Doug: having concrete proposal from roc will help resole that
… pace will accelerate dramatically
… will keep coordinating
Implementation status
Adam (Ericsson): Prototype implementation has been out for a while
Adam: We have a MediaStream implementation, getUserMedia with a
dialog, PeerConnection which currently opens a new port for every
new media (no multiplexing)
... We could do a demo, perhaps
anant: any particular challenges so far?
Adam: with the gstreamer backend, directly hooked up directly with
video source in webkit. this means we cannot branch out from this
pipeline to go in other directions
fluffy: has anyone done ICE? caps negotiation; dtls, strp, how far
along on these?
adam: using libnice for ICE based on gobject, part of the collabra
project
... no encryption or renegotiation
... prioritized codecs that we need to test with. using motion-jpeg,
h.264, theora
anant: are you looking to implement your prototype from a browser
perspective?
adam: yes
stefan: also looking to interoperate with internal SIP systems
harald: public initial release for chrome in november
... main missing pieces were a few patches waiting to land in
webkit, and some audio hacks to connect a media stream from the
peerconnection into an audio element (little cheating here and
there)
<fluffy> Public estimate of release is in November. Main bits
missing were some patches in webkit. Still have audio hack simple
put object from PeerConnection to speakers. Have ICE have encrypting
with lib jingle. Negoatiatoine crypto key. Have implemetation of
DTLS-SRTP but not yet hooked up
harald: we do have ICE, we do have encryption, using libjingle and
can negotiate crypto keys. have an implementation of dtls/strp but
not hooked it up into our webrtc implementation yet
... we are usign VP8 for video codec, and using tons of audio codecs
... not hooking up opus until we know what the IPR implications are
fluffy: SDP?
harald: using the old version of somewhat SDP proposed earlier
<fluffy> using VP8, waiting for IPR clarification before adding OPUS
harald: chrome on desktop only, not working on android yet
prefixing?
harald: yes, -webkit prefix
... patches are landing on the central webkit repository, as well as
chrome
... interesting trying to keep versioning straight amongst all these
repos
<juberti> i think the bulk of the webrtc code is in the embedder
section of webkit, so not directly included in what apple would use
On to tim, for mozilla
Tim's slides: [20]WebRTC implementation status in Firefox
[20] http://www.w3.org/2011/04/webrtc/wiki/images/4/42/Webrtc-mozilla-impl-status-20111101.pdf
tim: slides are similar to presented in quebec
... waiting for reviews before we land new libcubeb backend
... roc has an implementation of an audio API that plays silence,
which is progress since last update
... made more progress on the actual code
... list of open bugs on slide, using google webrtc code drop. work
on integration into that codebase
... no longer doing an add-on, doing a branch of firefox instead
... camera support working on linux and mac, build system issue
preventing windows for now
... dtls, strp are all getting started, moving echo cancellation to
hardware etc. no patches but hopefully soon
... Q1 2012 for a functional test build
???: thought about mobile?
tim: we have, number of challenges for mobile. no great audio API on
mobile, not low latency. need help from google before we can build
something people can actually use
harald: we (google) can't have a low latency API without low latency
support from android either
tim: we will probably hook up with a sound API initially (a demo)
that it's possible
opera: different timeline. we are pushing getUserMeida separately
from PeerConnection. labs build that implements getUserMedia is out
there, timeline to implement PeerConnection is not in the next 6
months
Incoming notifications
DanD: New agenda topic, seeking feedback on an open issue
... boils down to notifications.
DanD: You want to be called on your browser but your browser is not
up and running.
... Want to understand how big of a need this is.
... Need a platform-independent commonly supported push notification
framework.
... How does such a push notification reach a browser that isn't
running, and how does it start the browser and the correct web
application.
... If this is solved somewhere else in W3C, we need to have a note
that references it.
... May need something in the API to indicate if you want to send
such a notification when you try to start a session.
DanD draws diagram on the white board ([21]Fig 4).]
4 possible scenarios for incoming calls: 1) Web app is running, 2)
Web app is running but out of focus, 3) Web app is not running but
browser is running, 4) Web browser is not running.
Fig 4 - Incoming notifications
anant: Essentially related to the open question from yesterday: do
we allow incoming calls even when the user is not on a particular
website, but is online?
... Can we require the web app to be running in some form or
another?
DanD: Want it to work even if it's not running.
anant: Actually, I would advise against that.
... I want to be in a state where I don't want to receive calls.
DanD: But you set up your browser to receive notifications.
anant: But that's the same as always being signed into gmail. It's
an explicit user action.
fluffy: When you have a cellphone and you turn it on, it will always
accept incoming calls.
... I'm not suggesting the model of the past is the model of the
future, but I think we're moving more and more towards devices that
are always on.
anant: It's easy to build automated stuff on the web, so you're
going to get in trouble with spam.
... Just a note of caution.
francois: I don't get the difference between having a setting in the
browser and running a web app.
DanD: This is why I was drawing this diagram.
... If none of these things are running and I send a notification, I
want to be able to launch the browser and have contextual
information to say that a user of this browser wants to be able to
talk to you on app A.
francois: So you're outside of the context of the browser.
DanD: The browser may be in the background, but it might be not
running altogether.
... If I'm truly going to implement an incoming call use case,
there's no point if I can't answer that call.
RichT: There are four scenarios
... 1) The web app is running in the browser and focused
... 2) The web app is running but not focused
... 3) The browesr is open but not the web app
... 4) The browser is not running.
anant: You always have to have something, a small daemon listening
for notifications.
fluffy: I want to talk about the first one first.
... I think if you have 10 apps all running at the same time, and
they all have a TCP connection back to the webserver and they're on
a mobile connection
... you have to do a liveness check fairly frequently
... so your battery life is abysmal.
... If you do that in an app on the Apple app store, they will ban
your app and make you use their single global connection.
DanD: I captured that by saying that case 1 is solved, but
inefficient.
RichT: We have web notifications come out of a different working
group.
... if an app is not focused there should be an even that an
incoming call is received that can trigger a web notification.
anant: That part is missing from the API right now, and we should
add that.
... We need an API to register a callback when an incoming call
occurs
francois: It could be an event.
anant: Should it receive a PeerConnection object in that callback?
... Depends on whether we want to start doing ICE and reveal IP
address before answering.
RichT: We could trigger an OS level notification, but require the
user to click on it to take them to the webapp.
anant: Could we apply the same thing for case 3?
... I was thinking that when the OS notification is generated, you
click on it, and the web app is opened in a new tab?
hta: Is that in the spec?
anant: I'm just brainstorming ideas.
hta: I was wondering whether the notification API was part of the
callback API you were describing or if it was new things that you
think we should put into it.
anant: I think the notification API is open-ended.
hta: That kind of open-endedness is a null specification. It doesn't
tell me whether it's going to work or not.
DanD: The moment you rely on something that is OS dependent, how do
you know that it works interoperably?
... What is the level of support of this web notification?
RichT: It's still a draft, but it's part of the web, and should work
on all devices.
francois: In the 3rd case, the app is not running?
RichT: It's headless, so there's no UI.
DanD: There is still the situation where you don't have any headless
thing running.
francois: If nothing is running, how do you know where to dispatch
the incoming notification?
DanD: It has to be dispatched to the default runtime to kick start
the process of running the web app.
francois: Which app?
DanD: That has to be part of the notification.
hta: An incoming notification to my computer has no business knowing
which browser I'm running.
DanD: I agree with you. It should invoke the default web browser.
anant: What if that browser doesn't support WebRTC?
fluffy: Then you need to uninstall it :).
DanD: But I said in the notification I need to know what application
to launch to get back into focus.
... otherwise I don't know what JS to load.
hta: Basically you need a URL.
anant: But to generate that notification you're relying on non-web
technology.
francois: It means your listening to notifications that could come
from anywhere.
... Maybe it could be restricted from known domain names.
... Otherwise you could flood the user.
Stefan: I think you could register your web application with web
intents to be notified.
... something is left running even if you close the browser.
... To get from the server side to the browser we have something to
send events and web sockets, so that part is solved.
... Some part of the device must be awake and listening to pushed
events.
hta: Some devices like Chromebooks don't have the concept of
browsers not running.
fluffy: If you work in a web framework that's generic, you want to
be able to optimize it for lower power situations.
Stefan: I think web intents is similar to this notification idea.
You actually rely on the OS to wake up your platform, but don't
specify how it's done.
DanD: The only reason I wanted to have this discussion was not to
solve it.
... but to understand the reasoning, if it's a technology we have to
support, what the options are, and what the dependencies are for
future work.
Vidhya: Is one of the conclusions we're coming to, do we need an
application ID associated with the notification?
francois: No.
Stefan: We haven't gone into that level of detail.
DanD: We don't need an application ID, we need a way for the end
parties to register with a web server.
... There is some sort of a presence notion.
... How do you convey, "I know where you were registered last time,
how do I send you an alert?"
... If I go to Facebook, I know how my friends are, but I don't care
what application they use, as long as they're registered with a
service that will do the handshake.
... It's not that webapp A will send the notification, but there
will be a mediation framework that optimizes the notifications like
fluffy said and packages them to send the results.
... to determine if you need the user or the application or you need
both.
Vidhya: I feel uneasy because that ties you to a specific server.
This should be open.
DanD: I'll give you an exact use case. I called my mom from my
browser.
... She was on a physical phone, and she hangs up, and then wants to
call me back.
... Where is she going to call?
hta: It's not at all clear that she should be able to.
fluffy: We have this odd tradeoff between one connection for
everything you'd ever want to connect to.
... There's different levels you can aggregate on, from one
connection per site to only one connection total.
... They all make me very uncomfortable.
Vidhya: Once I got my iPad I noticed certain things that I couldn't
do without an iPad or without being connected to Apple, and this was
supposed to be web technology.
RichT: I don't think it's tied to a particular server.
... Browser B has established a connection to a provider, and the
provider knows where the browser is.
... the message is received with a particular context, and the
context decides what to do.
... I think it's just going to be a process of that out-of-band
channel.
DanD: It has one big assumption: that the browser is up and running.
RichT: If the browser isn't running, but we're still running a
context, we're fine.
... A JS context would provide the same functionality.
DanD: I'd like to learn more about this headless thing.
... From what I'm hearing you're saying it covers all the scenarios.
... It goes back to fluffy's point that these things may not be
efficient.
francois: What is not efficient may be the way that you push the
notification.
... Which is why I was talking about the server-sent event, which is
an ongoing draft.
... It's precisely meant for mobile devices.
DanD: The conclusion is we have a need, we may have solutions, and
we need to capture how we can deal with these scenarios.
... I'm trying to understand if there's anything we need to put in
the API to indicate if I want to send notifications.
anant: I was saying we need a callback for receiving them.
francois: Is that up to the app?
DanD: I'm done.
Stefan: I think we need to describe the use case.
DanD: I'd like to see some links to some of the works on IRC.
anant: Web notifications is still in editor's draft, so they're even
farther behind than we are.
<francois> [22]Server-Sent Events
[22] http://dev.w3.org/html5/eventsource/
<francois> [23]Web Notifications (exists both as public working
draft and editor's draft)
[23] http://www.w3.org/TR/notifications/
RichT: Should notifications be explicitly asked for, or something
that is always done?
... At the moment it's going to be an extra thing that developers do
voluntarily, but perhaps it has to be implicit.
DanD: From what I understand there's nothing that has to be built
into the browser to receive notifications.
anant: Web notifications needs to be implemented in the UA to be
useful.
DanD: So the work that is planned for web notifications is the hool
that is needed.
anant: I think they are a fairly good match.
hta: I think that the next thing that needs to happen is that
someone needs to look at these mechanisms and try to build what we
need.
... if that covers the use cases then we're done.
... if not we have to take notes of that.
Stefan: Are you volunteering DanD?
DanD: I don't have a working demo.
hta: I don't think you need a working demo of WebRTC, just to be
able to get web notifications going.
Stefan: Or just look at the specs and see if they do what we need.
DanD: I can definitely look into that.
RichT: If web notifications were to work, you're going to click this
thing and get back into the tab, but then get back into the tab and
show the normal accept UI.
... should we have a way to do this in one step?
hta: That depends on how much richness you're going to have in the
notification.
anant: You may need to be able to refuse, pick cameras, check your
hair, etc.
fluffy: When you're trying to filter spam, the consent message
itself can deliver the spam.
anant: You definitely have to think about spam.
fluffy: I don't know the answer, but it's a great point.
RichT: It's not perfect, but it does allow the focus to go back
naturally for the webapp.
anant: For the good calls, but we want to separate them from the bad
calls.
Any other business?
hta: We've gotten through some fairly hefty discussions.
... we have a few action items that are reflected in the minutes,
and some that are not.
... including an action item to come up with a specific proposal for
what a track is.
anant: One of the editors.
hta: We did not get around to saying how tracks, MediaStreams map
onto RTP sessions and all that.
Stefan: That's for the IETF.
hta: As long as they know what things we think they should be
mapping on to, they might have a better chance of hitting them.
... In two weeks it's Taipei, and we'll revisit some of these issues
at the IETF meeting.
... What other things have we forgotten?
fluffy: Just putting my IETF chair hat on, what things are useful
that came out of this meeting?
... We're okay with this ROAP model?
... We seem to be okay with DTLS-SRTP, we're still discussing
permission models.
... we talked about codecs.
... The long-term permission model.
... IETF is working on the assumption that we need the long-term
permission model, so that doesn't change.
anant: There's also the issue of identity.
hta: We have raised the concept that buying into a specific identity
scheme is probably premature.
anant: Agree.
hta: We seem to think that interfaces that allow us to tie into
identiy schemes are a good thing when we need them.
fluffy: If that's possible. Did that make it into the minutes?
derf: Yes.
anant: I think the right answer is a generic framework that you can
hook up any identity service, it was just motivated by browser ID.
fluffy: And ekr's draft gives two examples.
RichT: I was thinking that this could be done with DNS.
fluffy: So you call fluffy@cisco.com?
... I have some browser code for you...
anant: ikran. It's just SIP.
... Facebook may not have their user names in the form of
username@facebook.com.
... If we can build a more generic thing, then we can get buy-in
from more services.
... It's a balance between short-term how many people can we get to
use this service vs. having something interoperable, where the
e-mail address format is much better.
fluffy: I don't think it's the form of the identifier, but it's
clear identity systems are rapidly evolving.
... ekr has not claimed that we can have this pluggable identity
model.
... but he thinks it's possible and wants to go do the work, and
he's already shown it works for two.
anant: But that is very much an IETF issue, so definitely worth
discusssing in Taipei.
... though there's some overlap with W3C.
fluffy: IETF may say we're very happy to do the comsec security
issues, but we don't want to touch how do you display a prompt to
ask for a password, etc.?
hta: I thought it totally destroyed that distinction with oauth.
fluffy: I think it took the power of Lisa to make oauth happen at
IETF.
burn: It also needs to blend well with what web developers do.
... if they have to track too many identities it's a mess.
anant: If I want to write chatroulette.com, I don't want any
identity.
... and that should work, but if you want identity, you should be
able to get some assurance.
burn: But it should still be able to play well with existing methods
of identity on the web.
DanD: But if users say I want to have the identity of the other
party verified, they'll got with whatever solution is most
trustworthy.
anant: It depends on much trust the user has.
... The places where it may not be possible to use existing identity
schemes is when both parties don't trust the signaling service.
... Which is where we may not be able to use the existing identity
systems.
fluffy: That model is common (see OpenID, etc.).
... I think it's also interesting in oauth most of things you're
trying to solve can be solved with SAML, but the web hates SAML.
DanD: Most of the problems can be solved with long and short-term
permissions.
DanD: you might actually use oauth as a method of permission
granting.
Stefan: Any other final business?
hta: Thank you all for coming and well met at our next meeting?
DanD: When is the next meeting?
hta: After Taipei.
... is it reasonable to aim for a phone meeting in December?
fluffy: That sounds reasonable. I will be in Australia.
... I'm going to be up very, very late, aren't I.
hta: Somehow I doubt a doodle will end up with a convenient time for
Australia.
burn: Shall we take a poll to see how many people are sympathetic?
hta: Okay, so next meeting sometime in December timeframe.
... at that time we might have more input from the IETF and new
public WG drafts.
... we'll continue to make forward progress.
burn: They might not be public drafts, but that's more for
convenience and simplicity.
francois: Now that you've published your first public working draft,
you'll want to publish public drafts regularly.
DanD: Will the meeting notes be published?
francois: I'll clean them up and send them to the mailing list.
... I took pictures of the diagrams.
anant: I noticed the Audio WG was using hg.
... I'd like to switch to that, if it's not a big deal.
burn: We could put the draft on github.
fluffy: Are you okay with github, Adam?
AdamB: Yeah, yeah. I'm basically in favor of the systems in reverse
order of age.
RichT: You'll publish details on the list?
fluffy: This is just where we merge things together before putting
them in CVS, but we'll send details.
burn: Public drafts still need to use CVS, right?
francois: Yes, but that's not on your plate. That's internal W3C
business, because you don't have access to that CVS area.
hta: The editors will have to sort that out.
[meeting adjourned]
Summary of Action Items
[End of minutes]
Received on Tuesday, 8 November 2011 15:39:48 UTC