Simulation Results from the Adobe Dataset

Adobe provided us with data about the performance of their progressive font
enrichment system. I was able to use that data to run the simulations for
all of the different font transfer methods and then compare those to how
Adobe's system performs.

You can find the results of the simulations attached to this email. The
various transfer methods are scored on a two different metrics:

   1. Percent of bytes transferred vs. the optimal strategy. Lower is
   better.
   2. The total cost from the simulation (details on the cost function
   <https://docs.google.com/document/d/1kx62tpy5hGIbHh6tHMAryon9Sgye--W_IsHTeCMlmEo/edit#heading=h.4fz1x8661i63>)
   for each set of network parameters. Lower is better.

My takeaways from the data:

   - The 3 progressive enrichment strategies (Optimal, Patch subset, and
   Adobe's solution) all produce significant reduction in bytes transferred
   compared to current state of the art transfer methods. For example unicode
   range transferred 7.6 times more bytes than patch/subset.
   - Both Adobe's solution and Patch/Subset are pretty close to optimal
   with Patch/Subset transferring an additional 19%, and Adobe's solution an
   additional 45% versus the optimal transfer bytes.
   - Looking at the total cost for each method and network config it looks
   like PFE methods are usually beating unicode range and whole font methods
   except on connections with very high RTT's. This makes sense to me since
   PFE methods are typically going to result in more network requests. In a
   high RTT environment limiting the number of requests by using a single
   large font seems to be the winning strategy.

So overall these results are pretty good looking and validate that
progressive font enrichment approaches could materially improve the font
loading experience for users.

Some notes on the setup for the simulations:

   - As noted these simulations use exclusively CJK fonts. In particular it
   uses the set of open source CJK families available on adobe's service the
   majority of which are from either the Source Han Sans/Serif or Noto
   Sans/Serif CJK collections.
   - The network parameters used for the different network configurations
   are those defined here:
   https://github.com/googlefonts/PFE-analysis/blob/master/analysis/network_models.py
   - The cost function used for scoring the simulation results is the one
   here:
   https://github.com/googlefonts/PFE-analysis/blob/master/analysis/cost.py
   - All of the CFF fonts were de-subroutnized before being used in the
   simulation. This produced the lowest overall transfer sizes.

I'm currently working on running simulations on our own internal data set
that covers a number of languages beyond CJK. I hope to be able to publish
those results in the near future.