- From: Divakaran, Sudeep <sudeep.divakaran@intel.com>
- Date: Thu, 11 Mar 2021 03:12:37 +0000
- To: "'public-networks-ig@w3.org'" <public-networks-ig@w3.org>
Dear all,
We have created a github issue to track this topic
https://github.com/w3c/web-networks/issues/20
Members interested, kindly drop a note in this github thread or send us an email here. We can have a follow-up meeting to discuss how to take it forward (key topics, form a Community Group or Taskforce, etc).
Based on the feedback received below, some of these topics could spawn off as separate github issues like
- Network Trace Format specification
- Field trace capture tool and methodology
- A common repository (WPT or other) to store useful reference traces and classified in a standardized manner(subway, marketplace, InterRAT handover, switching networks, etc)
- Best ways to emulate network (within in browser, or reuse browser independent solution like throttle examples, netem mentioned below)
- Parameters to emulate (DL/UL BW, latency, location, other?)
- Recommendation on how to utilise this to aid web application development and testing (eg. in webpagetest.org, or mpeg-dash impl.)
- Enable consistent multiple browser support, define developer tools requirements, etc
Cheers
Sudeep
Co-Chair Web & Networks IG
-----Original Message-----
From: Dominique Hazael-Massieux <dom@w3.org>
Sent: Tuesday, March 9, 2021 8:35 PM
To: 'public-networks-ig@w3.org' <public-networks-ig@w3.org>
Subject: [minutes] Web & Networks Interest Group: Network Trace Tools, March 9 2021
Hi,
The minutes of our call today are available at:
https://www.w3.org/2021/03/09-web-networks-minutes.html
and copied as text below.
Dom
Web & Networks Interest Group: Network Trace Tools
09 March 2021
[2]Agenda. [3]IRC log.
[2]
https://lists.w3.org/Archives/Public/public-networks-ig/2021Mar/0001.html
[3] https://www.w3.org/2021/03/09-web-networks-irc
Attendees
Present
Ali_Begen, Ashish_S_Sharma, AutomatedTester, Barbara,
Chris_Needham, Dan_Druta, David_Burns, Dom, Ed_Lambert,
Eric_Siow, Huaqi, Jianjun, Jon_Devlin,
Jonas_Svennebring, Kenneth_Christiansen, krchrist,
Larry_Zaho, Neil_Craig, Peipei_Guo, Piers_O'Hanlon,
Piers_O_Hanlon, Song_Xu, Sudeep, Theo_Charitidis,
Yajun_Chen, Yoav_Weiss, Zoltan_Kis
Regrets
-
Chair
DanDruta, Song, Sudeep
Scribe
dom
Contents
1. [4]Intro
2. [5]Web & Networks IG charter
3. [6]Network Emulation Trace Tool For Browser
4. [7]Next steps
Meeting minutes
[8]Slides
[8]
https://lists.w3.org/Archives/Public/public-networks-ig/2021Mar/att-0001/Network_Trace_Tools_at_W3C_v1.0.pdf
Intro
Sudeep: today, we have guests from media & entertainment, Web
Performance, Browser Testing & Tools
Sudeep: main agenda today is a presentation on network
emulation tools for browsers, by Jonas
… he had given a presentation & demo in TPAC19
… he'll be sharing news on their project
Web & Networks IG charter
Sudeep: an updated charter for the IG is under review by the
W3C Advisory Committee
… until April 2nd
… please do bring this to the attention of your AC Rep
Dom: make sure to get your organization to chime in, important
to show support for the work
Song: ongoing discussions in Multicast / Broadcast in M&E;
network emulation can help on join work in W&N and M&E
Network Emulation Trace Tool For Browser
Sudeep: introducing Jonas, who presented Link performance
prediction before
… today, he'll present some of their work that enables
emulating network conditions from within a browser developer
environment
… including a proof of concept demo
… after the presentation, we'll have discussion on what
collaboration on this topic might look like
[Jonas showing slides [9]https://lists.w3.org/Archives/Public/
public-networks-ig/2021Mar/att-0001/
Network_Trace_Tools_at_W3C_v1.0.pdf]
[9]
https://lists.w3.org/Archives/Public/public-networks-ig/2021Mar/att-0001/Network_Trace_Tools_at_W3C_v1.0.pdf]
Jonas: this is a follow up to some of the topics we presented
at TPAC19
… on link performance prediction
… we have a number of internal tools to capture network quality
and play it back
… discussions in TPAC showed there is possible interest in that
tooling in a wider community
… we've reworked them and made them available as open source
… 3 type of tools: trace emulator, trace capture and a trace
format (allowing a library of existing network traces)
… these tools were built initially in the context of testing
link performance prediction and its usage in apps
… but the tools we're presenting here today are more widely
applicable, independently of prediction
… e.g. to show the impact of network quality variation on the
behavior of your app
… with a library of trace files, this lets you test your app in
very different network situations (e.g. NYC subway, driving in
bangalore, etc)
… On top of that, we have our performance prediction
capabilities
… We'll also touch on next steps
[10]Link Performance Prediction presentation at TPAC 19
[10] https://www.w3.org/2019/09/17-web-networks-lpp.pdf
Jonas: [slide 4] Our tools are collection of tools for chromium
browsers that gives you a visualization of network trace
… tool released on GPL2 - we're hoping to get contributions in
addition to re-use
… [slide 6] In terms of emulation, the idea is that you should
be able to get the experience as if you where in the
circumstances where the traces were collected
… A second aspect is to help apps leverage predictions to adapt
their behavior; useful e.g. in context of media streaming
… [slide 7] illustrates the network trace when testing
connection speed on fast.com
… as throttled by Chrome itself
… [slide 9] the capture tool is also available on github, can
be run from [11]https://intel.github.io/lpp-network-trace/
… you configure it with various parameters on the capture
session you want, e.g. frequency, bandwidth cap
… At this point, it's URL based; there are a bunch of speed
connection test available out there that we're thinking of
integrating directly in
… both end points of the connection matter in collecting the
traces
… you can also collect GPS data as you collect info, and keep
your screen on (important on mobile phone)
… mobile usage tends to provide the most interesting traces
… Right now, it's limited to chromium browsers
… [slide 10] once you start capturing, it generates JSON data
… that can be saved to the clipboard, which has worked well for
us
… [slide 12] The trace format is pretty-simple, JSON-based,
inspired by other W3C formats
… it starts with a header with general information, followed by
a sequence of entries that describe the network characteristics
and the capture circumstances
… the format is described in the github repo
… [slide 13: screenshot of a trace]
… format is flexible, e.g. no requirement to have both up- and
down-link
… [slide 15]: We've been collected traces into a library,
despite the limits imposed by the pandemic
… we're trying to collect traces from multiple locations,
multiple operators
… we welcome contributions on traces too
… [slide 17] if you're interested not just in emulation, but
also in prediction
… we have developed a set of algorithms to improve the UX based
on predictions of how network conditions might evolve
… with a particular focus on cloud gaming, media streaming
… e.g. prefetching more data when you expect the enter a
lower-bandwidth area
… these algorithms can be tuned e.g. on how much bandwidth
change should trigger a behavior change
… [slide 18] the github repo has an hello-world demo to show
how to pick up a prediction
… both pull & push approaches are available
… [slide 19] you get back a prediction with probabilities of
expected changes
… [slide 21] What is our roadmap?
… we're quite open - we'll be continue working on this, with a
plan to expand the emulator based on feedback we'll receive
from different usage scenarios
… in addition to downlink bandwidth, uplink, and latency are
good candidates
… other metrics are possible too
… more trace files is also in the plan
… we've extended the MPEG-DASH library to include LPP and
drive-tested it with various operators
… to demonstrate how it can improve streaming
… we're very interested in seeing if others are interesting in
driving this, e.g. as an MPEG-DASH extension
… our patches are still to be released, but we'd be happy to
start sharing them early before we complete our internal
processes for the full release
… we're also open to change how predictions get surfaced
… the remaining slides give background info on how to set the
tools up, and gotchas that you may face in that process
[11] https://intel.github.io/lpp-network-trace/
Sudeep: thank you Jonas
Ed: (from AT&T) capturing upload & latency would be great - is
it something already in progress?
Jonas: it won't be hard to do - we've been looking for feedback
& interest
DavidBurns: Browser Testing Tools WG chair - we're extending
how WebDriver work and are looking at network emulation
… e.g. to block requests to a CDN (who are bad when doing
automated testing)
… is this something that you'd be open for BTT to extend
Jonas: sure! we're not trying to drive this into a specific
direction
… we're happy to try and get these tools as useful as possible
… I'm open to any suggestion; open to figure out the right way
to collaborate
Piers: I'm assuming you're using the Web browser throttling API
to implement the emulation
Jonas: yes
Piers: have you calibrated that it gives the expected
performance?
Jonas: we've done a fair bit of testing
… on the emulator side, it has worked quite well
… the capture side has been more challenging
Piers: in terms of capture, you're using XHR - have you
considered using Fetch? WebRTC?
… for media delivery, you could look into capturing download
segments
Jonas: that's part of the feedback we're looking into for the
dash patches
… we haven't looked into that particular question
… quite a few aspects of network performance & metrics is app
dependent
… we're open to other ways to do this
… there are different ways of calculating bandwidth metrics
which may be of interest
… we'll invest more based on interest
Piers: what other techniques are using to measure?
Jonas: we have more complex tools that aren't part of the open
source release at this time, but may become so based on
interest; but the current tools are already pretty good
Song: The tool is meant to be used for mobile environments; I
had a quick test with the we-chat built-in browser - it could
be a very useful tool in the context of we-chat too!
… for the functionalities, I'm interested in seeing if that
tool can help in the discussion around broadcast / multicast -
I'll bring some of my colleagues in to look into this
Jonas: do please let us know of interest - that's going to be
key in further work on this
Barbara: have you done any testing with live streaming?
Jonas: are you referring to the dash extension or the chromium
browser?
Barbara: either - testing in the live streaming usage
Jonas: the trace collection is just raw data; when you go to
the emulator side, the actual usage doesn't really matter
… we have used the emulation while watching live streams
… but it's probably the dash.js aspects that makes the live
streaming use case more interesting
… we've spent a good amount of time on this; the patches
haven't been released yet
Barbara: broadcasters, online gaming are likely good use cases
for this
… In addition to WebRTC, WebTransport is another network usage
that is relevant
… I wonder if these traces could help in that space too
Yoav: If I understand correctly, the application of emulation
is based on the dev tools level emulation - is that correct?
… i.e. the same network emulation that Chromium dev tools would
trigger
Jonas: correct
Yoav: my personal is experience is that this type of emulation
is not ideal - it's done above the socket layer, which can
trigger various effects that are different from what would
happen from emulation happening at the network level
… it's a design choice Chromium folks did to avoid pollution
across tabs
… but it can have significant different on the results when it
comes to server behavior and prioritization
… it may be worth integrating with some network emulation tools
to better emulate what would actually happen on the network
Jonas: our internal tool operates at the network level
… but it's a lot more challenging to set up and deploy
… this is the trade-off between ease-of-use and accuracy
… I think I would be cautious with relying only with this
browser extension for validating considerations
… but it's probably already a significant improvement to what
developers would get exposed to by default
yoav: I agree that it's good context for development; for
testing, integrating with webpagetest.org a very popular
testing utility
… I don't know how easy it would be to integrate with that -
might be an interesting angle
… this would help maintain the ease of use while having more
realistic network conditions
… for testing that goes beeond the development
Dom: the trace format is independent of how the traces are
captured or at what levels the emulation runs, correct?
Jonas: correct
Next steps
Sudeep: good to see good suggestions and feedback
… there seems to be value in bringing that kind of capabilities
in browser dev tools
… also intersection with network predictions, usage in the
context of MPEG DASH
… I'm calling for people to express their interest, in terms of
possible improvements, additional traces
… is there a need for trace standard format standardization?
relation to the HAR format?
… with regard to adoption by web developers, Yoav's point on
webpagetest seems very relevant
… who would be interested in contributing to this?
intersections with other IGs/WGs?
… we could set up a dedicated CG to follow up on this if there
is interest
… a TF in this IG
… please send us an email if you're interested
Received on Thursday, 11 March 2021 03:12:56 UTC