[Update][minutes] Web & Networks Interest Group: Network Trace Tools, March 9 2021

Dear all,

We have created a github issue to track this topic 
https://github.com/w3c/web-networks/issues/20


Members interested, kindly drop a note in this github thread or send us an email here. We can have a follow-up meeting to discuss how to take it forward (key topics, form a Community Group or Taskforce, etc).

Based on the feedback received below, some of these topics could spawn off as separate github issues like 
- Network Trace Format specification
- Field trace capture tool and methodology
- A common repository (WPT or other) to store useful reference traces and classified in a standardized manner(subway, marketplace, InterRAT handover, switching networks, etc)
- Best ways to emulate network (within in browser, or reuse browser independent solution like throttle examples, netem mentioned below)
- Parameters to emulate (DL/UL BW, latency, location, other?)
- Recommendation on how to utilise this to aid web application development and testing (eg. in webpagetest.org, or mpeg-dash impl.)
- Enable consistent multiple browser support, define developer tools requirements, etc 

Cheers
Sudeep
Co-Chair Web & Networks IG

-----Original Message-----
From: Dominique Hazael-Massieux <dom@w3.org> 
Sent: Tuesday, March 9, 2021 8:35 PM
To: 'public-networks-ig@w3.org' <public-networks-ig@w3.org>
Subject: [minutes] Web & Networks Interest Group: Network Trace Tools, March 9 2021

Hi,

The minutes of our call today are available at:
  https://www.w3.org/2021/03/09-web-networks-minutes.html


and copied as text below.

Dom


           Web & Networks Interest Group: Network Trace Tools

09 March 2021

   [2]Agenda. [3]IRC log.

      [2]
https://lists.w3.org/Archives/Public/public-networks-ig/2021Mar/0001.html

      [3] https://www.w3.org/2021/03/09-web-networks-irc


Attendees

   Present
          Ali_Begen, Ashish_S_Sharma, AutomatedTester, Barbara,
          Chris_Needham, Dan_Druta, David_Burns, Dom, Ed_Lambert,
          Eric_Siow, Huaqi, Jianjun, Jon_Devlin,
          Jonas_Svennebring, Kenneth_Christiansen, krchrist,
          Larry_Zaho, Neil_Craig, Peipei_Guo, Piers_O'Hanlon,
          Piers_O_Hanlon, Song_Xu, Sudeep, Theo_Charitidis,
          Yajun_Chen, Yoav_Weiss, Zoltan_Kis

   Regrets
          -

   Chair
          DanDruta, Song, Sudeep

   Scribe
          dom

Contents

    1. [4]Intro
    2. [5]Web & Networks IG charter
    3. [6]Network Emulation Trace Tool For Browser
    4. [7]Next steps

Meeting minutes

   [8]Slides

      [8]
https://lists.w3.org/Archives/Public/public-networks-ig/2021Mar/att-0001/Network_Trace_Tools_at_W3C_v1.0.pdf


  Intro

   Sudeep: today, we have guests from media & entertainment, Web
   Performance, Browser Testing & Tools

   Sudeep: main agenda today is a presentation on network
   emulation tools for browsers, by Jonas
   … he had given a presentation & demo in TPAC19
   … he'll be sharing news on their project

  Web & Networks IG charter

   Sudeep: an updated charter for the IG is under review by the
   W3C Advisory Committee
   … until April 2nd
   … please do bring this to the attention of your AC Rep

   Dom: make sure to get your organization to chime in, important
   to show support for the work

   Song: ongoing discussions in Multicast / Broadcast in M&E;
   network emulation can help on join work in W&N and M&E

  Network Emulation Trace Tool For Browser

   Sudeep: introducing Jonas, who presented Link performance
   prediction before
   … today, he'll present some of their work that enables
   emulating network conditions from within a browser developer
   environment
   … including a proof of concept demo
   … after the presentation, we'll have discussion on what
   collaboration on this topic might look like

   [Jonas showing slides [9]https://lists.w3.org/Archives/Public/

   public-networks-ig/2021Mar/att-0001/
   Network_Trace_Tools_at_W3C_v1.0.pdf]

      [9]
https://lists.w3.org/Archives/Public/public-networks-ig/2021Mar/att-0001/Network_Trace_Tools_at_W3C_v1.0.pdf]

   Jonas: this is a follow up to some of the topics we presented
   at TPAC19
   … on link performance prediction
   … we have a number of internal tools to capture network quality
   and play it back
   … discussions in TPAC showed there is possible interest in that
   tooling in a wider community
   … we've reworked them and made them available as open source
   … 3 type of tools: trace emulator, trace capture and a trace
   format (allowing a library of existing network traces)
   … these tools were built initially in the context of testing
   link performance prediction and its usage in apps
   … but the tools we're presenting here today are more widely
   applicable, independently of prediction
   … e.g. to show the impact of network quality variation on the
   behavior of your app
   … with a library of trace files, this lets you test your app in
   very different network situations (e.g. NYC subway, driving in
   bangalore, etc)
   … On top of that, we have our performance prediction
   capabilities
   … We'll also touch on next steps

   [10]Link Performance Prediction presentation at TPAC 19

     [10] https://www.w3.org/2019/09/17-web-networks-lpp.pdf


   Jonas: [slide 4] Our tools are collection of tools for chromium
   browsers that gives you a visualization of network trace
   … tool released on GPL2 - we're hoping to get contributions in
   addition to re-use
   … [slide 6] In terms of emulation, the idea is that you should
   be able to get the experience as if you where in the
   circumstances where the traces were collected
   … A second aspect is to help apps leverage predictions to adapt
   their behavior; useful e.g. in context of media streaming
   … [slide 7] illustrates the network trace when testing
   connection speed on fast.com
   … as throttled by Chrome itself
   … [slide 9] the capture tool is also available on github, can
   be run from [11]https://intel.github.io/lpp-network-trace/

   … you configure it with various parameters on the capture
   session you want, e.g. frequency, bandwidth cap
   … At this point, it's URL based; there are a bunch of speed
   connection test available out there that we're thinking of
   integrating directly in
   … both end points of the connection matter in collecting the
   traces
   … you can also collect GPS data as you collect info, and keep
   your screen on (important on mobile phone)
   … mobile usage tends to provide the most interesting traces
   … Right now, it's limited to chromium browsers
   … [slide 10] once you start capturing, it generates JSON data
   … that can be saved to the clipboard, which has worked well for
   us
   … [slide 12] The trace format is pretty-simple, JSON-based,
   inspired by other W3C formats
   … it starts with a header with general information, followed by
   a sequence of entries that describe the network characteristics
   and the capture circumstances
   … the format is described in the github repo
   … [slide 13: screenshot of a trace]
   … format is flexible, e.g. no requirement to have both up- and
   down-link
   … [slide 15]: We've been collected traces into a library,
   despite the limits imposed by the pandemic
   … we're trying to collect traces from multiple locations,
   multiple operators
   … we welcome contributions on traces too
   … [slide 17] if you're interested not just in emulation, but
   also in prediction
   … we have developed a set of algorithms to improve the UX based
   on predictions of how network conditions might evolve
   … with a particular focus on cloud gaming, media streaming
   … e.g. prefetching more data when you expect the enter a
   lower-bandwidth area
   … these algorithms can be tuned e.g. on how much bandwidth
   change should trigger a behavior change
   … [slide 18] the github repo has an hello-world demo to show
   how to pick up a prediction
   … both pull & push approaches are available
   … [slide 19] you get back a prediction with probabilities of
   expected changes
   … [slide 21] What is our roadmap?
   … we're quite open - we'll be continue working on this, with a
   plan to expand the emulator based on feedback we'll receive
   from different usage scenarios
   … in addition to downlink bandwidth, uplink, and latency are
   good candidates
   … other metrics are possible too
   … more trace files is also in the plan
   … we've extended the MPEG-DASH library to include LPP and
   drive-tested it with various operators
   … to demonstrate how it can improve streaming
   … we're very interested in seeing if others are interesting in
   driving this, e.g. as an MPEG-DASH extension
   … our patches are still to be released, but we'd be happy to
   start sharing them early before we complete our internal
   processes for the full release
   … we're also open to change how predictions get surfaced
   … the remaining slides give background info on how to set the
   tools up, and gotchas that you may face in that process

     [11] https://intel.github.io/lpp-network-trace/


   Sudeep: thank you Jonas

   Ed: (from AT&T) capturing upload & latency would be great - is
   it something already in progress?

   Jonas: it won't be hard to do - we've been looking for feedback
   & interest

   DavidBurns: Browser Testing Tools WG chair - we're extending
   how WebDriver work and are looking at network emulation
   … e.g. to block requests to a CDN (who are bad when doing
   automated testing)
   … is this something that you'd be open for BTT to extend

   Jonas: sure! we're not trying to drive this into a specific
   direction
   … we're happy to try and get these tools as useful as possible
   … I'm open to any suggestion; open to figure out the right way
   to collaborate

   Piers: I'm assuming you're using the Web browser throttling API
   to implement the emulation

   Jonas: yes

   Piers: have you calibrated that it gives the expected
   performance?

   Jonas: we've done a fair bit of testing
   … on the emulator side, it has worked quite well
   … the capture side has been more challenging

   Piers: in terms of capture, you're using XHR - have you
   considered using Fetch? WebRTC?
   … for media delivery, you could look into capturing download
   segments

   Jonas: that's part of the feedback we're looking into for the
   dash patches
   … we haven't looked into that particular question
   … quite a few aspects of network performance & metrics is app
   dependent
   … we're open to other ways to do this
   … there are different ways of calculating bandwidth metrics
   which may be of interest
   … we'll invest more based on interest

   Piers: what other techniques are using to measure?

   Jonas: we have more complex tools that aren't part of the open
   source release at this time, but may become so based on
   interest; but the current tools are already pretty good

   Song: The tool is meant to be used for mobile environments; I
   had a quick test with the we-chat built-in browser - it could
   be a very useful tool in the context of we-chat too!
   … for the functionalities, I'm interested in seeing if that
   tool can help in the discussion around broadcast / multicast -
   I'll bring some of my colleagues in to look into this

   Jonas: do please let us know of interest - that's going to be
   key in further work on this

   Barbara: have you done any testing with live streaming?

   Jonas: are you referring to the dash extension or the chromium
   browser?

   Barbara: either - testing in the live streaming usage

   Jonas: the trace collection is just raw data; when you go to
   the emulator side, the actual usage doesn't really matter
   … we have used the emulation while watching live streams
   … but it's probably the dash.js aspects that makes the live
   streaming use case more interesting
   … we've spent a good amount of time on this; the patches
   haven't been released yet

   Barbara: broadcasters, online gaming are likely good use cases
   for this
   … In addition to WebRTC, WebTransport is another network usage
   that is relevant
   … I wonder if these traces could help in that space too

   Yoav: If I understand correctly, the application of emulation
   is based on the dev tools level emulation - is that correct?
   … i.e. the same network emulation that Chromium dev tools would
   trigger

   Jonas: correct

   Yoav: my personal is experience is that this type of emulation
   is not ideal - it's done above the socket layer, which can
   trigger various effects that are different from what would
   happen from emulation happening at the network level
   … it's a design choice Chromium folks did to avoid pollution
   across tabs
   … but it can have significant different on the results when it
   comes to server behavior and prioritization
   … it may be worth integrating with some network emulation tools
   to better emulate what would actually happen on the network

   Jonas: our internal tool operates at the network level
   … but it's a lot more challenging to set up and deploy
   … this is the trade-off between ease-of-use and accuracy
   … I think I would be cautious with relying only with this
   browser extension for validating considerations
   … but it's probably already a significant improvement to what
   developers would get exposed to by default

   yoav: I agree that it's good context for development; for
   testing, integrating with webpagetest.org a very popular
   testing utility
   … I don't know how easy it would be to integrate with that -
   might be an interesting angle
   … this would help maintain the ease of use while having more
   realistic network conditions
   … for testing that goes beeond the development

   Dom: the trace format is independent of how the traces are
   captured or at what levels the emulation runs, correct?

   Jonas: correct

  Next steps

   Sudeep: good to see good suggestions and feedback
   … there seems to be value in bringing that kind of capabilities
   in browser dev tools
   … also intersection with network predictions, usage in the
   context of MPEG DASH
   … I'm calling for people to express their interest, in terms of
   possible improvements, additional traces
   … is there a need for trace standard format standardization?
   relation to the HAR format?
   … with regard to adoption by web developers, Yoav's point on
   webpagetest seems very relevant
   … who would be interested in contributing to this?
   intersections with other IGs/WGs?
   … we could set up a dedicated CG to follow up on this if there
   is interest
   … a TF in this IG
   … please send us an email if you're interested

Received on Thursday, 11 March 2021 03:12:56 UTC