RE: Network Emulation Trace Developer Tools Proposal: PoC Info

Thank you Yoav, Tim, Peter, Patrick, Danilo for your quick feedback and thoughts.

We have now created a github issue to track this topic
https://github.com/w3c/web-networks/issues/20


Members interested, kindly drop a note in this github thread or send us an email here. We can have a follow-up meeting to discuss how to take it forward (key topics, form a Community Group or Taskforce, etc).

Based on the feedback received below, some of these topics could spawn off as separate github issues like

  *   Network Trace Format specification
  *   Field trace capture tool and methodology
  *   A common repository (WPT or other) to store useful reference traces and classified in a standardized manner(subway, marketplace, InterRAT handover, switching networks, etc)
  *   Best ways to emulate network (within in browser, or reuse browser independent solution like throttle examples, netem mentioned below)
  *   Parameters to emulate (DL/UL BW, latency, location, other?)
  *   Recommendation on how to utilise this to aid web application development and testing (eg. in webpagetest.org, or mpeg-dash impl.)
  *   Enable consistent multiple browser support, define developer tools requirements, etc

Cheers
Sudeep
Co-Chair Web & Networks IG

From: Tim Kadlec <tim@timkadlec.com>
Sent: Wednesday, March 10, 2021 8:10 PM
To: Peter Hedenskog <peter@wikimedia.org>
Cc: Divakaran, Sudeep <sudeep.divakaran@intel.com>; Patrick Meenan <pmeenan@webpagetest.org>; Yoav Weiss <yoavweiss@google.com>; public-web-perf@w3.org; Dominique Hazael-Massieux <dom@w3.org>; DRUTA, DAN <dd5826@att.com>; Song XU <xusong@migu.chinamobile.com>; Svennebring, Jonas <jonas.svennebring@intel.com>
Subject: Re: Network Emulation Trace Developer Tools Proposal: PoC Info

Super keen on digging in a bit more as this looks _very_ interesting.

I concur, how the developer can adapt the app based on these tests isn’t very clear. Perhaps examples like how NetInfo API are used by Apps today could be one.

FWIW, while obviously it's best if we can provide some sort of recommendations for immediate action from using something like this, I'm a big proponent of "surfacing leads to fixing". As in, over and over we see that greater visibility provided by improved metrics and tooling leads to significant innovation around optimizations. This feels like it could be one of those situations. I think the community has circled around what to do with network variability for a while and I suspect there are "things" that we can do there that better tooling could help us identify and suss out.

The collected trace could follow a standard format.

Yeah, this'll be key.  (I'll need smarter people than I to jump in with what those formats should be). Biggest impact is if this is something that WPT, SiteSpeed, Lighthouse, etc can all consume if they want.

Take care,
Tim

---------------------------------------------
https://timkadlec.com<http://timkadlec.com>
https://twitter.com/tkadlec<http://twitter.com/tkadlec>


On Wed, Mar 10, 2021 at 2:34 AM Peter Hedenskog <peter@wikimedia.org<mailto:peter@wikimedia.org>> wrote:
Hi,
it would be cool if the trace files could be used in https://github.com/sitespeedio/throttle. That way devs can use it independent of browser/tool and with more accuracy than using the built in network throttling that some browsers support.

Best
Peter

On Wed, Mar 10, 2021 at 7:09 AM Divakaran, Sudeep <sudeep.divakaran@intel.com<mailto:sudeep.divakaran@intel.com>> wrote:
Thanks Yoav for sharing possible ideas and connecting us with Patrick & Tim from webpagetest.org<http://webpagetest.org>. The suggestions are in-line with the discussions we have had in our Web & Networks IG (WNIG).

I think Patrick’s suggestion regarding wiring it to netem is a good recommendation too.

The collected trace could follow a standard format. How it is used for network emulation can be topic for broader discussion, e.g. either its done in the browser (current demo proposal) or even lower using netem, etc.

The greater interest seems to stem from the view that having the capability to emulate large variances in cellular connectivity from region to region (expected between & within networks) will benefit testing applications like media streaming quickly (ref traces from sub-way, driving through congested market place, InterRAT handover or while switching networks, etc are good examples). Having a common repo that the browser developer tools can use (like WPT repo) are indeed good suggestions.

I concur, how the developer can adapt the app based on these tests isn’t very clear. Perhaps examples like how NetInfo API are used by Apps today could be one.

To add, another topic being discussed in our IG, which is still under evaluation, is using predicted information about varying cellular conditions (if possible to make it available through some API). It may be better option to adapt, than relying on real-time congestion info or net info.

We will help connect the WNIG members with webpagetest.org<http://webpagetest.org> team to discuss this further (adding Jonas in the loop of this thread as well). We will also create a github thread so that we can track the discussions and documentations in one place.

Regards
Sudeep
Co-Chair Web & Networks IG

From: Patrick Meenan <pmeenan@webpagetest.org<mailto:pmeenan@webpagetest.org>>
Sent: Tuesday, March 9, 2021 9:58 PM
To: Yoav Weiss <yoavweiss@google.com<mailto:yoavweiss@google.com>>
Cc: Divakaran, Sudeep <sudeep.divakaran@intel.com<mailto:sudeep.divakaran@intel.com>>; Tim Kadlec <tim@timkadlec.com<mailto:tim@timkadlec.com>>; public-web-perf@w3.org<mailto:public-web-perf@w3.org>; Dominique Hazael-Massieux <dom@w3.org<mailto:dom@w3.org>>; DRUTA, DAN <dd5826@att.com<mailto:dd5826@att.com>>; Song XU <xusong@migu.chinamobile.com<mailto:xusong@migu.chinamobile.com>>
Subject: Re: Network Emulation Trace Developer Tools Proposal: PoC Info

Looks interesting.  I could see WebPageTest supporting a trace profile to use during testing - would just need to figure out how to wire it into netem.  The more interesting parts would be to better emulate the ramps for cellular connectivity.

That said, I'm not sure web developers would be able to do much with it. They tend to have enough of a problem optimizing for a few fixed profiles and adding variability to that may not result in useful signal for them to optimize towards (at least for the typical case of web page loads).  A video stream or protocol/congestion control development are much better fits for being able to adapt to variable conditions quickly.

I'm happy to chat more, otherwise I'll keep an eye on the project and see if there's anything I can expose in WebPageTest to help.

On Tue, Mar 9, 2021 at 10:28 AM Yoav Weiss <yoavweiss@google.com<mailto:yoavweiss@google.com>> wrote:
Hey folks,

To continue the conversation from today's meeting<https://www.w3.org/2021/03/09-web-networks-minutes.html>, I think it might be interesting for y'all to talk to +Patrick Meenan<mailto:pmeenan@webpagetest.org> and +Tim Kadlec<mailto:tim@timkadlec.com> who work on webpagetest.org<http://webpagetest.org>

Here's the scenario where I imagine network tracing to be extremely interesting:

  *   Folks around the world capturing real-life network conditions (e.g. "mid-day in Bangalore", "rural area in US", "London Tube", etc)
  *   WPT has a collection of those traces that developers can automatically apply when testing their site's performance
  *   Developers (automatically) test their site loading under those conditions, and (hopefully) find performance bottlenecks that are currently invisible
What do y'all think?

Cheers,
Yoav

On Mon, Feb 8, 2021 at 11:32 AM Divakaran, Sudeep <sudeep.divakaran@intel.com<mailto:sudeep.divakaran@intel.com>> wrote:
Dear Chairs and all in Web Perf WG,

I am Co-Chair of the W3C Web & Networks IG. At TPAC 2020 WG meeting, during the Network Diagnostics API proposal discussion, I brought up a couple of topics which may be of interest in this WG. One amongst them was regarding a proposal we received to extend network Developer tools to emulate time-variant network conditions to assist web app testing (e.g. either emulating real world trace or user defined traces).

Background:
I would like to follow up from where we left. The Web Performance WG Chairs expressed interest and asked us to share any links, when available.
Pls. refer to minutes <https://w3c.github.io/web-performance/meetings/2020/2020-10-TPAC/index.html#h.8gfnc7p4nih>
I quote - “Yoav: Network trace idea sounds interesting in context of synthetic testing, but possibly orthogonal to this. If you could share links that'd be great.”

Info:
The IG Members proposing this tool has made a simple Proof-of-concept, which also integrates into Browser Developer Tools (it’s a very basic implementation with just one parameter viz., DL bandwidth).
For more details, take a look at the demo video available on TPAC 2020 website @  <https://www.w3.org/2020/10/TPAC/wn-lpp.html>
Source codes and other info. are available on Github at  <https://github.com/intel/lpp-network-trace>

Upcoming Event:
We plan to have the team make a presentation in our upcoming Web & Networks IG meeting in early March ’21 (date yet to be finalised). We can share the Webex details, if anyone would like join us. Kindly let me know. I can work with the W3C Staff to forward the Webex Details, once the dates are finalised.

Request:
The team is seeking inputs and W3C engagement from interested members to add more features (emulate time variant latency, UL BW, etc), define trace format, Network Developer Tools UI, etc.
Our Interest Group is looking for feedback about this tool proposal and whether it fits in the broader scheme of things for Browsers. We would be happy to answer any queries using this email thread.

Regards
Sudeep
Co-Chair, Web & Networks IG
(sent on behalf of IG WNIG Co-Chairs & W3C Staff in CC)