- From: Dominique Hazael-Massieux <dom@w3.org>
- Date: Wed, 22 Nov 2017 14:41:39 +0100
- To: "public-webrtc@w3.org" <public-webrtc@w3.org>
- Cc: "public-media-capture@w3.org" <public-media-capture@w3.org>
Hi,
The minutes of our F2F during TPAC on November 6 and 7 are available at:
https://www.w3.org/2017/11/06-webrtc-minutes.html
I'm uploading video records of the meetings to media.w3.org and youtube,
and will link to them from
https://www.w3.org/2011/04/webrtc/wiki/November_6_-_7_2017 once that's
done; that being said, as experienced by the remote participants, the
quality of the audio is so bad that it's not clear how useful these
records actually are in this case.
Dom
[1]W3C
[1] http://www.w3.org/
WebRTC TPAC F2F
06-07 Nov 2017
[2]Agenda
[2] https://www.w3.org/2011/04/webrtc/wiki/November_6_-_7_2017
Attendees
Present
Bernard, AdamBe, DanBurnett, Huib, PeterT, JanIvar,
Varun, Cullen, Dom, hirokias, hiroki_asakimori,
Stefan_(remote), Jan-Ivar, Youenn, EricCarlson,
Hyukhoon_shim, PeterB_(remote), Soares
Regrets
Chair
Bernard, HTA (remote), Stefan (remote)
Scribe
dom, pthatcher, burn, DrAlex_, DrAlex, fluffy, adambe,
vr000m, soareschen
Contents
* [3]Topics
1. [4]WebRTC Testing
2. [5]Welcome / Intro
3. [6]Test as You Commit Policy
4. [7]WebRTC-PC Issues
5. [8]Issue 1635 Need for Initial Bitrate by the
Application/RtpSender?
6. [9]Issue 1194 audioLevel of tracks, both send and
receive?
7. [10]Adding more values to RTCIceTransportPolicy Enum
#1644
8. [11]behaviour of offerToReceive* set to false when
there is a local track #1586
9. [12]Isolated Media Streams requires modification on
permission algorithms in GUM and Permissions specs
#1646
10. [13]SVC Use Cases For an SFU developer
11. [14]WebRTC Stats
12. [15]Issue 177. Caching and consistency of getStats()
13. [16]Issue 131/PR 262. "objectDeleted" marker
14. [17]Issue 231/PR 273. Sender and Receiver stats vs
Track stats
15. [18]Issue 230/PR 272. RTCMediaStreamTracks to 4 dicts
16. [19]Issue 133/135. DSCP information
17. [20]Issue 202. concealedAudibleSamples
18. [21]Issue 1613. Stats and Isolated Streams
19. [22]Media Capture and Streams
20. [23]Content hints for MediaStreamTrack
21. [24]Clarify whether RTCRtpContributingSource members
are live. #1533
22. [25]RTCPriorityType combines relative bitrate with QoS
priority, which applications may not want. #1625
23. [26]Media Capture, WebRTC-PC and WebRTC-Stats Next
Steps
24. [27]WebRTC NV / New work
25. [28]Wrap Up Items
26. [29]Disable user media by default in cross-origin
iframes #434
27. [30]Audio-video RTC over QuicStream
28. [31]Scary ICE proposal
29. [32]Wrap up and actions
* [33]Summary of Action Items
* [34]Summary of Resolutions
__________________________________________________________
<hta2> I don't see anyone on the testing webex - are you in the
meeting webex?
<fluffy> stefan, are you on IM here ?
<dom> hta2, can you say something?
<fluffy> IM us back as we are not hearing you
<hta2> Sorry, not looking at all the screens all the time while
waiting - did you get the sound check for me OK?
<inserted> ScribeNick: dom
WebRTC Testing
<hta2> Sound from the meeting room is very low.
<lgrahl> Ouch!
<hta2> Sounds like you have too many devices with the
loudspeaker on in the room.
<hta2> if someone can turn the camera to point at the speaker,
that might be nice.
Huib: I work for Google; used to work for Opera and have been
working with Web technologies for a long time
... have been working on finalizing the WebRTC technology and
we have been working with testing to that end
... making the Web work reliably is a key aspect of providing
APIs ot the Web
... WebRTC 1.0 has been a long road, with many specs both APIs
and protocols at W3C and IETF
... lots of efforts to get there
... Web Platform Tests is a great tool for testing Web
technologies, but it is not sufficient for WebRTC given the
complexity of WebRTC, and the number of underlying low level
protocols
... So Google took up the idea of Alex to work on a more
comprehensive system - KITE, an open source project we formally
announced last week
... Alex will tell us more about it
... Beyond spec and testing, the third part if compliance - all
browsers have their own challenges in that space
... We've made progress in Chrome in the past year - compliance
on getStats, constraints, RTCRtpReceiver
... The major upcoming milestone is unified plan, planned for
Q1 2018
... getting to 1.0 is critical to avoid getting too many
proprietary or native solutions deployed for WebRTC
... The major engineering effort in WebRTC in Chrome is around
reliability - which is much harder than spec compliance
... other browsers are probably in the same situation
... Soares will report on his work on WPT, sponsored by Google
Soares: quick update on the status of compliance testing in WPT
... We have ~1300 tests - added 1000 in the past few months
... We have added a tool to calculate coverage for WebRTC
<hta2> Repeat: If it is possible to turn the camera towards the
speaker, that would be good.
Soares: We extracted all the normative statements
<hta2> Thanks!
Soares: the WebRTC 1.0 spec is about 70% tested at this stage!
<lgrahl> Can you talk louder? Sound is cut off a lot.
Soares: Some of the challenges to reach full coverage
... tests are currently written based on the June editors draft
- quite a few updates since then, so the tests are a bit out of
sync
... some race conditions make test difficult
... WebRTC describe steps that happen in parallel - sometimes
hard to trigger specific concurrent operations
DanBurnett: you noted 10% untestable assertions - what's your
thinking on that?
Soares: the untestable ones are bound to the constraint of what
is exposed in plain JavaScript
DanBurnett: if these assertions come from the spec, the spec
may need to be changed to avoid those
Alex: Some errors can't be dealt with at the JavaScript level -
e.g. errors linked to hardware error which can't be triggered
... re race conditions, dealing with timing and promise-based
algorithms is hard
Jan-Ivar: we may need to fix the spec if the timing is not
deterministic
Alex: re untestable, we count them as "tested" in the 70%
... regarding the new 97 steps added since June, many of them
come from error reported out of the testing effort
... in the remaining path toward full coverage, 90% of it is
easy; we expect to reach 100% in Q1 2018
... once we reach 100% for webrtc-pc
Soares: note that coverage doesn't mean they are all correct -
the tests need to be verified by implementors and editors
... one possible topic is how to keep track of coverage over
time
... we're looking at automation with web driver & permissions
API
... there are other specs that need testing efforts: we are on
track on testing mediacapture-main next - other specs are
further down in the pipeline
[Slides: Real-Time Communication Testing Evolution with WebRTC]
Alex: beyond testing the JavaScript compliance, there is a lot
more that is needed to properly test an RTC system
... JS API compliance is part of the standardization process -
but the tests are standalone tests, testing a single browser at
a time
... Until WebRTC, this was probably sufficient
... but for WebRTC, you need tests that test interop across
browsers
... you also need to test intersection with protocols and the
networking layer (e.g. ICE, NAT traversal)
... So beyond compliance testing (with automation, stand alone
tests, one browser), we need to move to interop testing (still
with automation, but on two synchronized instances of the test,
with 2 or more browsers)
<lgrahl> Sorry for nagging but sound is cut off a lot again. :/
Alex: For automating interactions, Web Driver helps quite a bit
... But we need improvements there - e.g. extensions to
integrate permissions management in Web Driver
... Once you've obtained permissions, you need ways to emulate
captured media
... browsers have already ad-hoc ways to do that, but we want a
standardized Web driver approach to it
... So Web Drivers help quite a bit, but it doesn't help when
it comes to run synchronized tests across browsers
... lots of tools hvae been developed to address part of the
problem - which confirms the problem exists and is important
... how bad is it when you try testing interop across browsers
/ OS - leading to ~500 situations to test
... only very few of these situations had been automated so far
... existing solutions have limited reach in terms of browser
versions and OS
... eg in the enterprise market, Windows 7 is still a key OS to
test for
... different browsers have different level of support for Web
Drivers, and even more so when taking into account the
permissions extension
... to avoid regression tests or ensure newer versions of
browsers are as compliant / interoperable as possible, bugs
need to be caught early in the release process (i.e. in
~nightly builds)
... depending on resources, your needs on
freedom/convenience/price vary quite a bit
... Taking all these dimensions into account, what you can
achieve with existing solutions is always limited
... Testing WebRTC feels like death by overwork, hence we
created this interop testing engine named "Karoshi Interop Test
Engine", aka KITE
... it is designed to be as generic as possible
... meant to work with any client, any platform, any back-end
... we architected this by designing 4 independent components:
a selenium grid, a test engine, tests themselves, and a
reporting dashboard
... tests and dashboard aren't completely independent since the
dashboard needs to know what to expect from the tests
... Tests have to encapsulated into a Java class
... configuration files are used to describe which tests are
run in what browsers, on what selenium nodes
... we also record timing information to detect performance
issues
<lgrahl> Sound (sorry) :)
[demo of KITE dashboards and how they allow to look into
details of runs]
scribe: This level of testing has already allowed us to
identify and report bugs
... we have plans to extend this to mobile browsers (will come
to open source project), to electron and other native platforms
(in the cosmo proprietary version)
... we will bring additions for SFU testing over time
... we are very open to all suggestions
... we want to make this tool as useful as possible - please
file issues on our github repo
Cullen: first, thanks, this is amazing
<hta2> the sound has become worse and worse over the period of
the presentation, with mike cutting out randomly. Now Cullen's
voice is much better than Dr. Alex' was.
Cullen: what are the next steps to expand this
Alex: Web Driver handling is key; support in Edge is important
to reach 100%
... Tests for IETF protocols are a big piece of the picture
<hta2> the new mike sounds better, at least at first.
Alex: a key topic for people is call quality
<hta2> but now it's doing the same thing again.
Alex: media quality is another topic - but hard to measure
automatically
... What I want is have a tool that moves beyond compliance to
quality, incl network and media resilience
... As far as standardization is involved, the most difficult
part will be around the number of protols
Cullen: the key is identifying key use cases - it's impossible
to test all the situations covered by the RFCs
Huib: from Google side, we've sponsored the development of KITE
to enable the proper testing of interop testing
... getting the important use cases exposed is key
... we want the baseline scenarios to be right
dom: how do you write an actual test case?
<lgrahl> The mic seems to work decent if the person talks loud.
Alex: we need to improve the documentation on this; this is
upcoming in the next few weeks
dom: how hard/easy is it to run WPT tests in KITE?
Alex: we're looking into making it easier
<hta2> now the sound is completely gone.
Huib: but this not the main use case for KITE - the expectation
is that this will run in the WPT dashboard
<lgrahl> Thanks dom for summarising - sadly I couldn't hear
much. :)
[coffee break until 10:45]
Welcome / Intro
[reviewing slides]
Test as You Commit Policy
[35]Adopt test as you commit policy
[35] https://github.com/w3c/webrtc-pc/issues/1617
Philip: I work from Stockholm Google office on testing
... WPT is big
... WebRTC test suite is in pretty big shape
... To keep the test suite in good shape, I have a proposal to
keep them in sync as the spec evolves
... We have an annotated version of the spec based on test
coverage rwaldron.github.io/webrtc-pc/
... the coverage of that spec is one of the better known of the
platform
... The WebRTC test suite is run daily on the WPT dashboard
wpt.fyi
... currently has lots of red, but most of it is linked to
automation of getUserMedia (or lack thereof)
... both for permission prompt and fake media generation
... so this should get much greener soon for Chrome and Firefox
soon
... we have a new piece of infrastructure to have tests to
drive themselves via Web Driver
... we plan to extend the Web Driver API to manage permission
prompt and to bring that to the WebRTC test case
... the WebRTC test suite is being imported into browser test
suites (at least Chromium, Gecko, Webkit)
... also test case updates get automatically upstreamed
... My proposal today relates to an approach groups have
started to adopt
... where a normative change to the spec comes with an
associated pull request on the test suite
<pthatcher> Sure...except my slides are next.
Philip: we hope to increase the production of tests based on
that policy - this is already having a clear impact
<pthatcher> Also.. .when I do scribe, do I have to be as
detailed as you? I don't know if I'm capable of typing that
quickly.
Philip: we have passed 50% of specs who have adopted the policy
danburnett: there is a discussion in the issue - e.g. what to
do if you don't know how to write test case
... also, we should understand if/when that policy applies -
for CR, it makes plenty of sense; for new work, there may be
too much change early on
Philip: the proposal here is to do this for WebRTC 1.0 spec
... some groups have decided CR is a good time to enforce this
... but clearly it can't wait until CR
Cullen: at a conceptual level, sounds great
... the practicality of it is a bit worrisome - will it slow us
down in closing bugs? it will improve quality for sure
... but maybe before adopting it, we should play with it to see
how it works out
Philip: note that the policy is not about adding a hard
requirement so much as encouraging it as a practice
... this may slow down the spec work, but would increase the
rate of interop
Youenn: fixing the platform is more important than fixing the
spec
... this simplifies implementation in browser vendors
<inserted> ScribeNick: pthatcher
Dom: how many people have experience writing tests cases, among
people who make changes to the text?
Cullen: I don't
(a few hands saying they do?)
foolip: the person who wants it merged is going to want to see
the test added
... but the test can be written by someone else
dan: Idea for policy: for a PR to be merged, there must be a
test. But the editors can make a judgement call to be able to
merge without a test.
foolip: This is like tests in a code review. It's a cultural
change.
Harald: feels safe to try this;
... if we don't try it, we won't get there
jan-ivar: What's the granularity?
Harald: I'm just talking for volume testing purposes
foolip: ... Can't be adding huge new things; so shouldn't need
huge new tests for small changes
... just small tests for small changes
dan: maybe we should wait until we have a certain test coverage
threshold
Bernard: a call for consenus on issue 1617 to provide a test
for changes unless you convince the editors it doesn't apply to
you
lots of thumbs up
no thumbs down
RESOLUTION: We adopt a test as you commit policy as describe in
issue 1617 - we may revisit if it proves too complex in
practice
Noted: a consensus to move ahead, with Cullen's coveat that we
can revisit this if we find it doesn't work well
foolip: I'll go write that PR
I can't scribe any more :)
WebRTC-PC Issues
<dom> ScribeNick: dom
[36]RTCPriorityType combines relative bitrate with QoS
priority, which applications may not want
[36] https://github.com/w3c/webrtc-pc/issues/1625
PeterT: we discussed as an interim before
... right now, you can't control separately priority and QoS
[37]Adding relativeBitrate parameter to
RTCRtpEncodingParameters
[37] https://github.com/w3c/webrtc-pc/pull/1632
Peter: the PR provides a more granular control on
RtpEncodingParameters, splitting bitrate and QoS priorities,
with a more flexible control on bitrate
Varun: relativeBitrate is that related to what?
PeterT: to other streams
Varun: so the top bitrate is one?
Cullen: there was confusion about rate being used and max
PeterT: same as currently, but with more control on the
relative values
Harald: I worry about this
... what you're saying now is that if the congestion point is
at the point of having DSCP glitches, you won't change anything
... e.g. if you want to prefer audio over video and get into
congestion, the relative bitrate will not get you what you want
(e.g. throwing video)
petert: if we focus first on splitting qos vs bitrate - what's
your opinion?
hta: from a process perspective, the IETF specs have already
passed last call, so that's a concern
... from a practical perspective, the risk is that we open up
many cases that make no sense for users that wouldn't do the
right thing
PeterT: there are 2 type of users: those that don't care and
those that do
... having one knob helps the former, having two helps the
latter
Harald: the relative bitrate control seems useless to me; but
changing the spec of control we already agreed upon worries me
Bernard: exploring one of the cases that match this question:
audio conf with low priority file transfer; but there is no
reason to keep its bitrate low
harald: the current spec says that when there is congestion,
the low priority @@@
stefan: we should have much more data about actual usage
nowadays
... I would like to see proof of need of this
... to get data on how this actually behaves
PeterT: we have an application that wants to use this field
... but when trying to implement it, the fact that the two are
combined was blocking us
Varun: we came across this as well
PeterT: we want to turn on DSCP markings to experiment with
them, but we can't do that if this is coupled with bitrate
Cullen: assuming we would agree to do this (i.e. it's not
useless), what would we need to change in specs?
<foolip> who wants to merge
[38]https://github.com/w3c/webrtc-pc/pull/1657?
[38] https://github.com/w3c/webrtc-pc/pull/1657?
Peter: so imagine we have one 4 level knob for QoS, and a 4
level knob for bitrate
Cullen: I don't think such a change would have to impact the
RTCWeb-transports draft
HTA: RTCWeb transports is where 1/2/4/8 is defined
PeterT: I'm happy with only doing the split of knobs - the
relative bitrate thing is just a nice to have
<hta2>
[39]https://tools.ietf.org/html/draft-ietf-rtcweb-transports-17
#section-4
[39]
https://tools.ietf.org/html/draft-ietf-rtcweb-transports-17#section-4
Cullen: I'm not against it; I'm not sure I'm convinced yet
PeterT: so the model would be to keep the current knob, and add
two "experts" one for separate control that would override the
other
Cullen: splitting the knobs would impact the rtcweb-transports
draft
PeterT: Let's look into this offline and come back to this
tomorrow
Issue 1635 Need for Initial Bitrate by the Application/RtpSender?
[40]Need for Initial Bitrate by the Application/RtpSender?
#1635
[40] https://github.com/w3c/webrtc-pc/issues/1635
Varun: the IETF congestion control wanted lots of info to help
with initialization
Cullen: I think the IETF would have concerns about this being
misused by attackers in JavaScript
Bernard: RMCAT could send a liaison statement if they want to
push for the W3C WG to adopt this
PeterT: my recommendation was pushing this for post 1.0
Varun: that's fine by me - but I want to emphasize that people
are complaining about the lack of current optimal starting
situation
... I'll reach out to the RMCAT WG to let them know they should
a formal request
PeterT: but even if and when they do, it's not clear where we
would put it
Bernard: we'll put that issue on the icebox for now
<fippo> fluffy: won't those security concerns apply to what is
currently possible in chrome with sdp munging?
Issue 1194 audioLevel of tracks, both send and receive?
[41]audioLevel of tracks, both send and receive? #1194
[41] https://github.com/w3c/webrtc-pc/issues/1194
PeterT: this can be done with Web Audio
Cullen: but not when the stream is isolated
PeterT: recommendation is not adding this
Bernard: no objection
Adding more values to RTCIceTransportPolicy Enum #1644
[42]Adding more values to RTCIceTransportPolicy Enum #1644
[42] https://github.com/w3c/webrtc-pc/issues/1644
PeterT: 2 different issue - one about network permission
separate from getUserMedia, the other about giving more control
for the app to constraint their routing
Varun: use cases including routing data through or not VPN
... another thing is to enable a relay-first approach to help
with establishing a call quickly
Dom: so if we don't do this, what should we recommend to the
OP?
PeterT: so for the first bit, we need to look into the
permissions thing
... for the second, give more justification / use case
Cullen: maybe a way to address the second thing is a way to
expose from which route the origin was loaded
PeterT: it may make sense to close this issue and open one on
data-channel only permission, and then maybe one discussion
exposing more routing info for selecting ice candidates
behaviour of offerToReceive* set to false when there is a local track
#1586
[43]behaviour of offerToReceive* set to false when there is a
local track #1586
[43] https://github.com/w3c/webrtc-pc/issues/1586
<fippo> i probably owe a PR here. Its a mess. (dom: note that
so its on record :-))
JIB: this case would only happen in the case where one of the
party is not a browser
RESOLUTION: we don't change how offerToReceive*: false behaves
in the spec
Isolated Media Streams requires modification on permission algorithms
in GUM and Permissions specs #1646
[44]Isolated Media Streams requires modification on permission
algorithms in GUM and Permissions specs #1646
[44] https://github.com/w3c/webrtc-pc/issues/1646
Soares: discovered thing while testing isolated media
... right now, the way the permission grant doesn't take into
account the fact that permission is granted not to an origin,
but to a peer
DanBurnett: this needs to be described in webrtc not
getusermedia
Dom: but getusermedia probably needs a hook in its permission
check
... so I guess we do need to modify the specs
Soares: I can take a stab at it
Alex: who will review?
Dom: DanB+JIB for getUsermedia, MartinT & EKR for webrtc, …
SVC Use Cases For an SFU developer
Bernard: this is input to our discussion tomorrow on WebRTC NV
... we're doing it today due to time constraints of the
presenter
Sergio: this is meant as a conversation trigger for WebRTC NV
[slide SVC Use cases]
[slide Content Adaptation (II)]
Cullen: decoding complexity seems a bit different from the rest
... e.g. based on codec support
... e.g. it might be that stereo videos would use different
layers for the 2 eyes
Sergio: the most clear need is turning on/off temporal vs
spatial scalability
PeterT: I think we have everything in RtpEncodingParameters,
except for the spatial/temporal switches
Varun: there is also the question of priority - which layer
should switch to which priority
... not entirely sure if that level of control is needed
Cullen: I agree with these requirements
... when it comes to image size, we care about the thresholds
... we code to specific sizes for specific phones
... right now we work around this via stats
... but it would be more convenient to have direct control over
this
DanB: This all looks great; but how doable is it? how
standardized are the controls across SVC codecs?
... every codec seems to be doing it a bit differently
... can we define a predictable outcome?
PeterT: I believe if we just add the high level switches for
scalability (temporal and spatial), yes
Sergio: agreed; it would be hard to do low-level control
because of this variability, but high level control seems very
feasible
Bernard: there is an implicit assumption in what PeterT said:
that a decoder can always decode what an encoder sends
... that's true for VP9, but it may not be true for all
current, let alone future codecs
PeterT: if you set the scalability bit, if the codec doesn't
support it or if the recipient can't support, then you just use
simulcast
DanB: that's what's at the heart of my comment - can this
actually be implemented?
cullen: there aren't so many SVC codecs, and they've all been
designed similarly - so yes, this should be doable
Bernard: AV1 has a similar syntactic structure simliar to H264
... it may break the assumptions of VP8/VP9 of encoder //
decoder
<burn> scribenick: burn
WebRTC Stats
[Varun presents slides]
<dom> [45]WebRTC Stats Issues (30 atm)
[45] https://github.com/w3c/webrtc-stats/issues/
Issue 99/PR 251: Security and privacy considerations
s;Issue 99/PR 251: Security and privacy considerations;Topic:
Issue 99/PR 251. Security and privacy considerations;
Dom: for the security-impacting features in Stats that are
beyond what webrtc-pc itself introduces, we will send the APIs
to the TAG with a proposal
Bernard: "network layer data" is vague. I suggest describing
"statistics related to packets lost/received, etc."
Varun: counters might be better
<dom> Cullen: some of the stats exposed here can fingerprint a
specific network based on its characteristics, which can help
determine two peers are on the same local network
Varun: regarding location access, we need feedback
Cullen: what can you do with this that you can't do with HTTPS?
Dom: by exposing rtt between peers, given the location of one
you may be able to infer location of the other
HTA: can tell distance between clients, not just clients and
servers
Dom: and these stats can be used without the user knowing it.
Data channels alone with this could be used to triangulate.
Varun: and this is useful for CDNs.
Bernard: candidate pairs also give you RTTs.
Dom: we will expose the problems that exist and discuss
possible mitigations
Next step would be to merge this PR (after Varun updates).
Issue 177. Caching and consistency of getStats()
<dom> [46]Discuss caching and consistency of getStats() return
#177
[46] https://github.com/w3c/webrtc-stats/issues/177
Varun: PR 243 is related
JIB: PR says when you cannot cache
Dom: since this says For Example, the MUST should be lower case
Issue 131/PR 262. "objectDeleted" marker
<dom> [47]Consistent marker for "non-active" object? #131
[47] https://github.com/w3c/webrtc-stats/issues/131
<dom> [48]Added 'objectDeleted' attribute #262
[48] https://github.com/w3c/webrtc-stats/pull/262
JIB: if we keep around stale data, under what conditions would
they go away?
<dom> [49]https://github.com/w3c/webrtc-stats/issues/235
[49] https://github.com/w3c/webrtc-stats/issues/235
Varun: this came up before, but we haven't seen any problems
with it. Issue 235
<dom> [50]Is keeping stats around a memory problem? #235
[50] https://github.com/w3c/webrtc-stats/issues/235
Varun: we haven't seen it, but others might.
JIB: Why can't we just grab stats just before closing an
object?
HTA: sometimes an object is destroyed by remote control, e.g.
setRD that closes some of the connections.
Varun: also a third party might be monitoring and not be privy
to the actions taken by the application.
JIB: what about a time limit?
HTA: for a long time now the limit has been the life of the PC
... we could add an API for stats clearing.
Adam: what about an API to register a capture for anything you
would care about?
Varun: if you are a third-party monitor this is too hard to
make work
JIB: this is a slippery slope. Maybe we need something else in
reality, like a session ended event. Let's solve the right
problem.
HTA: we have event transitions in theory. Maybe in NV we can
consider other options I will send by email.
JIB: FF doesn't work like this, so we want to make sure this is
necessary before we do it.
Dom: does that mean issue 235 should be a CR blocker?
JIB: we will review and get back to you.
... this is related to sender vs. receiver stats
Varun: back to 262, should we do it?
JIB: that's where we had concerns
Varun: if 262 depends on 235, which is in icebox, we have a
problem
Dom: if either blocks CR we need to discuss.
JIB: yes, we need to review and propose an alternate
Dom: are we marking 235 as CR blocking and linking it to this
issue?
(agreement)
and Jan-Ivar owes a response to it.
Issue 231/PR 273. Sender and Receiver stats vs Track stats
JIB: using replaceTrack will replace the counters on the sender
side but not on the receiver side
... builds on next topic of Issue 230/PR 272.
... idea is to have counters on senders and receivers.
varun: not having track info there is a problem with
replaceTrack because track stats go away in this approach. We
need the stats to stay around.
JIB: this sounds CallStats specific. In general we have assumed
that with a PC we don't get a done event for every action.
Varun: intercepting WebRTC APIs is problematic.
JIB: this is not an issue for original JS app code, since their
code causes the actions and thus knows.
... instrumentation is an edge case, albeit an important one.
... how often do we need to know the counts before and after a
replaceTrack?
Varun: switching front to back cameras, or screen capture, are
very good examples.
JIB: getting tough to parse stats now, since you have to know
which objects have been deleted.
Varun: we should separate the two issues / the two PRs. Getting
an event would be nice, but we hear users asking to be able to
call getStats at the end, after objects have been ended.
... this has not been an issue so far because we haven't had
lots of tracks lying around. Hard to know how this will change
going forward.
JIB: the issue seems to be removing track stats, right? Is
there a problem with adding the sender and receiver stats?
varun: not at all. It's the removal of the track stats that
would be an issue.
JIB: okay, let's focus on the addition first.
varun: +1
JIB: I will update the PR
Issue 230/PR 272. RTCMediaStreamTracks to 4 dicts
JIB: this is editorial only since the API is unaffected.
Varun: no objections. What would be required now if we get
depth stats?
JIB: same as what we have. Might be a bit more prose.
Cullen: how does extensibility work?
Varun: easier with an enum.
JIB: depth is simpler because it inherits off the existing
dictionaries
HTA: another extensibility piece to track.
JIB: any objections?
Varun: let's make sure this is JS observable.
(no objections)
Issue 133/135. DSCP information
Cullen: (asks about how it would look)
HTA: we have two alternatives because it's not really clear why
someone would prefer one over the other
Cullen: maplike is easier to use, but a string could be parsed
as well.
HTA: (missed comment on strings). With maplike you would have
to iterate.
Dom: any privacy or security issues?
... say, bleaching of JS?
Varun and Cullen: yes, but there's not much you know.
JIB: record is closer to a JS object so might be passed by
reference, but don't know for sure.
(agreement this is useful)
(but not on which to choose)
JIB will review and comment
Issue 202. concealedAudibleSamples
Bernard: why during audible portions?
Varun: if concealment is noise or content is important.
Cullen: there is an RFC that defines concealed.
Bernard: in XR block
Varun: but it doesn't say whether it's in an audible section or
not.
... packet loss, audible sample, or not are the three factors
Bernard: RFC 7294
Varun: there are many issues like this that are not
controversial but are lacking expert review.
Cullen: we were only supposed to reference existing stats and
not create new ones, so maybe these need to be removed if they
are unclear.
... we were only supposed to add stats that are clear to the
spec editor.
Dom: and this is not a CR blocker? why not?
Varun: virtually everything in getStats is optional anyway. We
may just provide a date cutoff for something going in.
... as soon as all the CR blocking items are in we will move on
even without these iffy items going in
Dom: should warn people of what might go in/not go in.
Varun: conclusion is we need a citeable reference that all
parties agree on.
Dom: Cullen is right that the experts are not likely in our
group.
Cullen: this specific metric has problems.
... the concealment algorithm introduced audio, but that's
about all. Get IETF to define the details if you really want
it. Concealed seconds is useful, but the high/low energy
distinction is of questionable use.
Varun: this is in a grey area where I don't see the value but
that doesn't mean it isn't useful to someone else.
hta: we have lost samples that occur in the middle of speech,
so you might want to replay rather than filling in.
... typical suggestion by someone but we can't decide whether
to refuse or find a better definition.
burn: the burden needs to be on the suggester.
Dom: let's ask for a more specific justification and reference.
Issue 1613. Stats and Isolated Streams
<dom> [51]Stats and Isolated Streams
[51] https://github.com/w3c/webrtc-pc/issues/1613
HTA: we know this can be an issue. So either we try to
mitigate, or we just remove the stat. The latter is my
suggestion.
Cullen: security arch doc actually says to do that.
HTA: I will prepare a PR for not reporting the stat in this
case.
Bernard: an issue for csrcs as well
... in webrtc-pc
Dom: will the PR address CSRCs as well, or just volume in
isolated streams?
HTA: i will do the stats part.
Dom: I will file a new issue for CSRCs
HTA: I will do the PR then.
Media Capture and Streams
<dom> ScribeNick: DrAlex_
<DrAlex> jib speaking
<DrAlex> starting with jib tickets in the absence of peter
sometimes the values that were preset are different from the
current values
example, capture rate might fluctuate up to a preset of 60 fps
live volume vs actual volume settings
is another example.
camera motor pan has a target, and a current value as well.
cullen: let s take th pan example
we need both values
jib: yes but one is not a constrainable
cullen: agreed
dan: we already discussed this
this should be what it is SET to
hence the name.
we can still discuss wether we also need a current value or
something, but the result of getSettings() should be what i was
SET too
bernard: the downside is that if we had a current value, people
would start pooling it.
jib: the spec has special frame rate wording.
dan: there may be a precision discussion we may have across
all measurements.
that is also true for framerate
which can fluctuate around a target velue
but your setting is what it is, and doe snto change.
cullen: it s ok to report error if you make a mistake.
or if there is a problem.
you should not set constraint, but when you do, and use exact,
you are expecting errors when things is not exact.
camera rates do not fluctuate. However browsers get frames at
different rates.
jib: cameras, e.g. on mac, have modes. if you ask for an
arbitrary rate you night not get it.
it is my opinion JS should count the frame rate themselves.
cullen: i do not think there is a way for app to compute FPS,
really.
jib: ok, so for motor pan .....
do we speak about target, settings, values
?
cullen: actually: what do we have again ?
<inserted> ScribeNick: DrAlex
issue 466
the answer is "it shoul dbe whatever the last setting was"
for previous issue.
<stefanh> was there any conclusion on 470?
<stefanh> audio is really bad
conclusion of 470 the answer is "it shoul dbe whatever the last
setting was"
<dom> (i.e. the target setting, not the live value)
settings shoul now belong to the track
<stefanh> +1 to hta's comment
of course source have settings, but source can have multiple
settings, from the user point of view (and also for security
reason) the settings should be on the track
jib
it would be very easy to see the track as a rescuer object.
should the track only reflect the source settings
or shoul dit handle all the scaler.
dan: when we spoke about it initially, we had decided that the
sink would be the one holding the processing (scaling here)
jib: we have now additional sink: canvas capture, remote track
.. we have never came up to define constraint on those. e.g.
echo cancellation is audio only .....
with that, I think that if we can word this in such a way that
tracks can have different settings we are still all in
agreement
cullen: isn t it like that already ?
consensus on leaving it as is
<DrAlex_> harald: we decided last time we had two ways to do
it.
<DrAlex_> one sends nothing
<DrAlex_> one sends black frame.
<DrAlex_> cullen: while I think we should specify it so people
know what to expect, but it s an implementation detail.
<DrAlex_> sergio: in the case of simulcast
<DrAlex_> we also have that kind of problem, when we kind of
"mute" the higher resolution in case of congestion. In the IETF
there is a solution.
<DrAlex_> bernard: i just looked it up. there is nothing about
enabled=false
<DrAlex_> ericsson: it does not matter if it s muted or
enabled=false, it should be the same behavior.
<DrAlex_> <everybody agrees>
<DrAlex_> jib: so ... no action item ?
<DrAlex_> cullen: two different settings. one is resulting in
silence / black to be send the second one stops rtp packets.
<DrAlex_> bernard: track=null results in no rtp sent.
the track could be enabled, while the source be muted.
(jib)
adam: the specs says balck frames and silence shoul dbe send.
but it does not specify wether you send only one such black
frame, one per second, or full stream of black frames at 30fps.
youenn: safari sends one frame per second, just to make sure
something is coming.
so does firefox.
<audio setting in the room>
jib: do have a motion toward consensus on this.
consensus
495 / 472 Video dimension constraint
<stefanh> What was decided on 441?
441: one frame per second.
scenario: sender has 16:9 receiver wants 1:1
<stefanh> BTW, the "one frame per second" was an advice, right?
shall we add pixels (letterbox) or removing piixels (crop)
proposal: new constraint resizeMode
what a bout default
cullen: ok, what abou tnone?
peter: not allowed to do any resizing
cullen: ok, love it.
peter: there is still the question of default
cullen "trash my video"
jib: my one concern is that we are moving to a "rescaler API"
we argued shoul dbe on the sink. if we re speaking about access
to the source ......
i like crop'n scale
cullen: i originally though like you. but justin convinced me
that for screen sharing cannot let any pixel to be thrown.
jib: different spec
adam: surveillance video also need to keep all pixels.
jib: ok
peter: to you point that it s become a size and rescale API,
yes, and that s what we aim at.
Dan: i do not think it conflicts with anything else.
dom: one confusing thing, there are many different constraints.
<stefanh> FWIW I like Peter's proposal 'resizeMode'
dom: we should start making the distinction between the
constraints used through applyConstraint, and those used in
GUM, ....
<dom> objectFit
<dom> +1
Harald: we should reuse the names from CSS
peter: agreed. i just could not remember them when i wrote
those slides.
harald: i ll dig them up
jib: so ....
the implementation concern is not w3c"s priority, but .... for
browser vendors, there are risks.
peter: this should ONLY be used for aspect ratio.
if the aspect ratio does nto change, it is not intended to be
used.
<stefanh> my understanding was that resizeMode would be used
also when width/height was constrained
jib: we might end up rescaling twice then
peter: yeah i guess
jib: do we want a 4th value "preserve aspect" as per cullen
suggestion?
peter: i ve never heard anybody asking for that.
jib: any other comment ?
consensus in the room
dan: the conversation was going fine, so i was DL the audio
system info
jib: is there a pr
<dom> consensus to add a resizeMode constraint
peter: no, not yet
<dom> JIB will produce a PR
<stefanh> consensus on default mode?
jib: ok, i ll write the PR.
<stefanh> crop-and-scale?
jib: with the new constraint we have a way forward, but we
still need to define a default.
cullen: constriant has a mechanism: minimize a istance function
youenn: no scaling at all, so we would prefer minimize
processing.
jib vs cullen round 5
on constrint.
constraint
proposal: unless people specifically requires resizeMode, the
default is to let the browser choose
consensus
<dom> Chair: Bernard_Aboba, Stefan_Hakasson_(remote)
<dom> Agenda:
[52]https://www.w3.org/2011/04/webrtc/wiki/November_6_-_7_2017
[52] https://www.w3.org/2011/04/webrtc/wiki/November_6_-_7_2017
<dom> [53]Slides
[53]
https://docs.google.com/presentation/d/1Sg_1TVCcKJvZ8Egz5oa0CP01TC2rNdv9HVu7W38Y4zA/edit
<dom> ScribeNick: fluffy
Content hints for MediaStreamTrack
On slide for "Issue 478"
Questions abotu what to do. Not many questions asnwered since
last time
This would be a new field on track or something like that
Seems where we are is people previously thought this was
generally a good idea but wanted to see specifics before they
agreed to accept or not
Argument for putting it in a track was that it needed to go to
multiple things, like recorder, so that made it better in track
and sender
<dom> cullen: another question is whether we have enough and
the right categories
<dom> ... e.g. having 30 (sport, ...) vs 2 modes, this is very
different conversation
Main driving use case was indicate that video should be treated
like screen cast
Question does this go on tracks or sinks
Concerns raised about auto setting as this implies it is a hint
not a setting
PeterB: If you make an app that does things like echo
cancelation and voice enhancements, they mess up things like
music
... browser has no sane way to set sane defaults for music vs
speach
... as more things get added to the browser, the music app will
need to know to turn it off if we don't have something like
this
Question about is auto value of enum or antoher flag.
Dan: We have an "auto" setting for constrainable properties.
This would allow the browser to use the seperate hint about
media type (music/speach) to do this
Adam: propose that is what we have today and this is what no
constraint means
Dan: OK, perhaps don't need auto
Agrement in room we do not need to add Auto
Question about if put it souce / sink
JanIvar: lean towards source as it is descriptive of the track
Adam: we don't have controll of all the sinks so better not to
it there. The fact that it might look like a camera, but it is
a source. So want to put on a Track.
Proposal put it on track. (and be great if we could hear this
from Randal if he is OK with this)
Questioon what happens if you have a track and you change the
value of the hint.
Dan: as long as browser is operatinng with constraints, it can
do what it wants and use info inculding this hint to decide how
to process it
JanIva: Set echo cancel false. Then set hint to voice. This
will not chang echo cancel as it was explicitly set. But if it
had not been explicitly set, then this hint might chang the
echo cance
Question about impact on GUM and extentiosn that don't know
about it
Dan: Do we need to say anything about what extention specs must
say? and other thing how does this change implemetation of
existing specs.
Adam: we will have some hints, and if you exptend the base
spec, you start with an empty bucket of items and so there is
no impact of adding this
Dom: we should have words on if you are a sink you need to
consdier the impact of hints
... do we try and bring that into GUM 1.0 or if this is a next
version thing
... answer depends on who plans to implement
Bernard: one place this made a big differnces is when we do AV1
which has screen content encoding - prob bit longer than 6
months
Question how would conflict setting be addressed
Dan: There may be different places where you set the track. If
you set it once and set it again, it replaces the previos. How
much info carries to toher side. Questions on where you can set
it.
JanIvar: any idea that this would cary across RTP over a clone.
Bernard: the decode knows what tools were used to encode but
does not need to set the hint. Or the app might know from JS to
turn it on. Would not assume it is carried over.
Dan: what about clone
DOm: wes should copy hint
JanIvar: this is a weak hint and you send it to web audio, hint
gets dropped
... if there was no processing, the hint would be lost
<burn> ouch
We need to avoid slipper slope of speach/child
Dom: how to we deal with timeline of changes to attribute and
when it happens to media
PeterT: no gaurentee of when it applies. It's a hint
What are audio category
speach and music seems to be the winners
Dom: so does speach and music corespond to what we need to
adjust.
... is there a ref set of categories we could use from
elsewhere to guide this
fluffy: existing codec have speach and music
Agrement in room: speach and music
Moving on to question to video categories
Bernard: even thought video came from screen, if it was a game,
it might be motion not slides
<dom>
[54]https://wicg.github.io/mst-content-hint/#video-content-hint
s uses "motion" and "detail"
[54] https://wicg.github.io/mst-content-hint/#video-content-hints
Adam: are we dupliating existing constraints ?
Fluffy: propose motion, animation, detail
<burn> (sorry about the echo - trying to find it)
Bernard: might effect the pallet used for encoding
Room is good with having at least motion detail
Question do we add third category of animation
Varun: will this confuse people
... will people know what detail is
Dom: we will have to provide good definitions
<stefanh> webrtc-pc talks about framerate/resolution
[55]https://w3c.github.io/webrtc-pc/#dom-rtcdegradationpreferen
ce
[55] https://w3c.github.io/webrtc-pc/#dom-rtcdegradationpreference
Dan: if input is a movie, don't change white balance or de
nosie
... but if it is a camera would be good to do
Varun: what if person says want motion and detail
Dan: think abotu this as use case "this is what is going
wrong", and thus "this is the hint you need"
<burn> varun asked what about motion and no-motion
<burn> then Peter said no-motion doesn't need a codec
Fluffy: think abotu example with comptuer vision and no codec
Adam: why do need the hint
Dom: hint is only usefull if there is something delegated to
the browser
Adam: can we do this already
PeterT: for many things yes, we have direct control, but for
some new things we don't have, having a hint helps
Varun: hints should not be on get user media
<dom> [56]Accelerated Shape Detection in Images I mentioned
earlier
[56] https://wicg.github.io/shape-detection-api/
JanIvar: the gum makes a track, and then things would allow
setting it
... setting low framerate is not good way to say prefer high
res
Varun: in screen capture of games, want high fps and good
detail
JanIvar: track sits between soruce and sink. It is a way to
configure source but it is also may have to become an API for
sync too - for example downscale
Back to quesiton on do we add thrid animation category ?
Dan: What is the use case
Fluffy: arch fly through with synthetic artificial images
Varun: because it is articfical mostion, we want to see all of
it in games and such
PeterT: had asked what you do because of this
... hint will not cover all use cases
Fluffy: would turn off pre processing of white balance,
de-noise, etc. Would pass differnt paramters to AOM1 codec.
soares: want a way to turn off much of the pre and post
processing
varun: want fine things or don't mess with motion
Adam: we are duplicating what might have in constraint
Fluffy: this is a "do what I mean" not "do what I said"
interface
Dom: hard to know we need for do what I mean. Less convinced
this model is good
<stefanh> What would the default be between motion and detail?
"balanced"?
Dan: sugest we need more work on why this is needed
PeterT: two things going forward. 1) recovince ourselves we are
OK with this model 2) do we want to add a third
JanIvar: like to see some examples of things we use this for
that we don't want to do with direct controlls
Varun: people do not want to figure out the the constraints and
instead just leave it empty and have the hint do the right
thing
<dom> [we need a boolean "doTheRightThing" constraint]
PeterT: uefull to have more specific of how an browser might
implement this and how apps might use it
<burn> [a Do What I Mean API is perfect, of course. If we had a
Jarvis we could ask him to create that API]
Moving on to Issue 1533
Clarify whether RTCRtpContributingSource members are live. #1533
<dom> [57]Clarify whether RTCRtpContributingSource members are
live. #1533
[57] https://github.com/w3c/webrtc-pc/issues/1533
JanIvar presenting: We could get live objects where you get an
object and get live values of it.
This does not work becuse you don't get new particpats. And get
old values for person that left.
Conclusion: live objects are not usefull
fluffy: why did we make it an interface
Adam: so do we have to poll
JanIvar: yes
... used to be one field, but now two. Why did we split
Dom: to make it cleaner ???
Agreed no live objects
Moving on to splitting priority up to two controlls
RTCPriorityType combines relative bitrate with QoS priority, which
applications may not want. #1625
PeterT: made PR to deal with this
<dom> [58]RTCPriorityType combines relative bitrate with QoS
priority, which applications may not want. #1625
[58] https://github.com/w3c/webrtc-pc/issues/1625
<dom> [59]Let javascript set different priorities for bitrate
and DSCP markings. #1659
[59] https://github.com/w3c/webrtc-pc/pull/1659
dom: prefer clean break
JanIvar: is there an impact on data channel
Bernard: think it is OK
Fluffy: on behalf of HTA, do we have people that set it
seperate
PeterT: Yes, we willhave people that set marking but not
bitrate
<burn> Cullen: and we will need to set bitrate.
Varun: prefer ratio bitrates
... would use bitrate prioriyt with the 4 values
Dom: was trying to get to if the bitrate priories in 4 bin was
worth doing now and would used. Is there enough value that we
do this
Vaarun: fine with 8 4 2 1 locking
Summary: If the IETF is ok with peters changes, then WebRTC is
in favour of making this changes
<adambe> scribenick: adambe
Media Capture, WebRTC-PC and WebRTC-Stats Next Steps
bernard: 15 issues relates to peer identity
fluffy: a lot of issues on peer identity are bogus
... no one is working on it
burn: Who's working on implementing (peer identity?)
bernard: We should look at interop requirements
fluffy: Our current text about two implementation is wrong
... it's more correct to say that there will be more than one
browser
dom: Is there a difference?
bernard: It's also a question about how code is integrated in a
browser
dom: The JS API is a small part of this
... What we want to verify is that it actually works to send
media between browsers
burn: It's also important for independent people to be able to
to implement the spec
bernard; Our edge-stack is fundamentally different
fluffy: It's hard to define independent implementations is our
case; and it doesn't buy us much
dom: We could probably get away with only testing the JS API,
but I think we have a moral duty to do more (i.e. interop on
media level)
bernard: Do we need to change our text in the spec about
"independent implementations"?
burn: We need to file an issue to update this text
AP on burn to file an issue
fluffy: It's in our charter that we should demonstrate media
interop
bernard: Moving to other specs in the group
... We should have wider review of screen capture spec
... We need to check with the "depth" editors on the status
... The major action items are around screen capture
jib: Several browsers have some kind of implementation of
screen captuer
<dom> [60]Screen Capture editors draft
[60] https://w3c.github.io/mediacapture-screen-share/
burn: The API should be standardized
fluffy: That's the easy part; the hard part is the privacy
aspects
bernard: We should include "screen capture" spec in the interim
meetings
AP on chairs: Bring screen capture in to the interm agendas
Topic is: WebRTC stats status
varun: summarizes the current issues
soares: It's quite straight forward to test the stats stuff
... We just need to put in the work
dom: Have you considered testing the life cycle of stats
objects?
... We don't need to check the exact value of metrics. Just
that they are not zero
varun: It's not easy to do
... That seems like browser testing to me
fluffy: It would be really nice to at some point test a bunch
of stats between a set of browsers and verify that the result
makes sense
varon: That kind of activity could be part of the "ultimate
goal"
jib: Make sure we distinguish between API tests and "browser
functionality tests"
... What about non-MTI stats?
varun: We're focusing on the MTI-ones at the moment
... Do we want allow experiments with stats?
fluffy: we can't disallow it
... Don't use prefixes on names
varun: Google has an old and a new API. Experiments are
currently done in the old API
Discussion about experiments with origin trials
scribe: and different approaches on doing experiments with
stats
<dom> ScribeNick: vr000m
WebRTC NV / New work
fine grained control for media, SVC, simulcast support.
Cullen: talking about Simulcast support.
...: we have level of simulcast support today, however, we do
not have everything.
Peter Thatcher: for the server to negotiate simulcast, for the
endpoint to receive simulcast (i.e., run the transcoding bridge
on the browser)
Cullen: Fine grain media stack control, we could take a high
level api is the pc spec, today. The ORTC stuff is moving away
from SDP, but it still follows the O/A.
...: so we should expose all the minimal features to the
javascript.
Pether Thatcher: ideas on WebRTC next steps
Cullen: can do codecs via webassembly should explore that
Alex: Explore the idea of using EME for hardware crypto
PeterT: QUIC recap
...: QUIC features for datachannels
... QuicTransport
... QuickStream discussion about ordered and unordered
Adam: there should be a way to get an event when data is acked
on the send side
jib: did you use "async for" from whatwg streams
peter thatcher: there are some roadblocks.
<jib> "async for" == for-await-of
[61]https://github.com/tc39/proposal-async-iteration
[61] https://github.com/tc39/proposal-async-iteration
DECISION: Add a constructor to the RTCIceTransport to the
WebRTC CR
Peter can write other docs for Quic and other changes
<soareschen> Cullen: How to make ICE easier?
Wrap Up Items
<dom> ScribeNick: soareschen
Jan on issue 434
Disable user media by default in cross-origin iframes #434
<dom> [62]Disable user media by default in cross-origin iframes
#434
[62] https://github.com/w3c/mediacapture-main/issues/434
<dom> [63]Feature Policy
[63] https://wicg.github.io/feature-policy/
Chrome has different syntax for iframes, i.e. allow="microsoft"
<dom> [64]allow attribute
[64] https://wicg.github.io/feature-policy/#integration-with-html
Jan: We should change the language in the spec for feature
policy
Related issue: 440
Adam: If Chrome is implemented that way and mozilla is
considering, we should revisit that
Dom: agreement
... Do mozilla expected to do both?
Jan: just the attribute
Dom: we can still move to PR if implementers already align with
the changes
... reopen issue 440
Jan: does absence of allow='microphone' means disallowed?
Dom: we could just reuse the sandbox
Peter: more scary proposal
<dom> Dom: I'll make sure to follow up internally to see if
feature policy can move to a more formal status
Audio-video RTC over QuicStream
way to get audio/video over any transport
Peter: low level way: stick in a track and get out a video
frame
... for decoder, provide a frame, and at future time will go
into a sink
Cullen: the frame need to include details such as timestamp
Does this mean the stream need to be ordered?
Peter: only needed for key frames
Dom: there are requests getting this feature into service
workers
Vr00m & Peter: flexibility of inspecting metadata of encoded
frames
Vr000m: Supporting SVC over QUIC?
Cullen: JS is fast enough to do SVC in JS
Vr000m: more metadata such as bitrate and target
... we can get a lot more control over this
... doing simulcast?
Peter: We could have 3 decoders
... We can get extra metadata such as layer for simulcast
Vr000m: Is it only one frame one frame?
Peter: You can have your own framing scheme for multiple frames
in one stream
Cullen: When decoding one frame may be decoded into many frames
Peter: Haven't thought through for multiple frames
Dom: dictionary format for codec metadata
... are only codecs exposed are codecs that can be specified as
dictionary?
Peter: there are two ways, not sure which one is better
Vr000m: on jitter buffer
Peter: you can have control with your own jitter buffer
implementation
in JS
Vr000m: want to control jitter buffer underneath to send only
one frame
Cullen: we can inspect timestamp metadata
Peter: we can specify frame dependency
Vr000m: if there is a jitter buffer underneath I want more
control over it
Peter: example for audio encoder/decoder
... audio jitter buffer is different. Sink is pulling at
regular rate regardless of whether it is filled
... if there is nothing you'll get silence
Vr000m: For video, we want to pass partial frames to video
decoder
Peter: for media over QUIC streams, we can experiment more
easily
without having to standardize first
Dan: did you talk about isolation?
Cullen: we can't have isolated media streams for this
Peter: one way is that the encoder can encrypt
Cullen: Or we can have isolated JS that does decoding
Jan: Audio worklet can run limited JS in audio thread
... Decoding video on main thread isn't desirable
Peter: technically we can implement encoder/decoder and jitter
buffer in JS
only difference is you can't get access to native or hardware
implementations
Cullen: You can build your own jitter buffer in JS, but most
would use a default native implementation
Peter: option B: high level send/receive example
... require mapping down to QUIC stream
Dom: Youenne have some security concerns
Cullen: if isolation is a concern, can tell the encoder to pass
only encrypted SRTP packet
... For decoder, pass SRTP packet to it which connect directly
to sink
... People who use this API probably care less about isolation
<dom> [65]Media Source Extensions
[65] http://w3c.github.io/media-source/
??: People might say we don't need another JS interface for
audio/video pipeline
Peter: does WebAudio has encoders?
No
Scary ICE proposal
Cullen: Scary ICE proposal
... API to send STUN requests
... All the stuff stay the same. On the other hands we have API
to nofify STUN requests
Things complicated for ICE is that we have two peers that test
connection independently
Cullen: Move complexity of ICE to JS. no need to do it
billaterally
... no need check list and gatherer state
... JS shouldn't able to modify STUN packet content
... browser would keep track of best one found
... but JS can decided which one to use
... we can find paths that ICE can't currently find
vr000m: Is it because current ICE have different priority?
Cullen: we don't want to allow JS to construct custom STUN
packet to exploit the server
Peter: as long as there is limitation on how to send STUN
packets it should be fine
... if we call send too fast, browser just return error
Cullen: browser always send STUN response to STUN packets
Peter: we also want notification responses
Cullen: problem of ICE is it only discover 2-way paths, not one
way
... enterprises often have firewalls and want 1-way path
Peter: there are strict rules on how much/fast you can send in
ICE, and we would still make those apply
Cullen: we are very careful to keep the security properties of
ICE
Peter: we follow the ICE rules for punch etc
Cullen: I am even willing and able to write a test for this
Wrap up and actions
Bernard: wrap up
Cullen: for encoder/decover we prefer option A over B
Peter: we want to write an ICE transport doc
Adam and Dan: the testing PR #1657 has been merged, and there
is now a testing.md with help
Bernard: do we have conclusion on SVC?
Cullen: we agree to not add new functionality for current spec
Dom: we should consider one document for each new feature
... we need new documents for QUIC stream and ICE transport
extension
... encoder/decoder might have much more impact beyond WebRTC
Cullen: the proposal would be have a subset for the ICE
protocol in IETF, then a W3c document for API access
Project snowflake for new ICE protocol
that's all for TPAC!
Summary of Action Items
Summary of Resolutions
1. [66]We adopt a test as you commit policy as describe in
issue 1617 - we may revisit if it proves too complex in
practice
2. [67]we don't change how offerToReceive*: false behaves in
the spec
[End of minutes]
Received on Wednesday, 22 November 2017 13:41:51 UTC