Re: [proposal] the Spotlight Project

[adding the HTML5 performance task force in the loop]

On Fri, 2013-05-24 at 18:16 +0000, Austin,Daniel wrote:
> Hi Team,
> 
>  
> 
>                The Spotlight Project is intended to provide insights
> into the overall user performance of Web pages by making test results
> public. In conversation with Phillipe earlier this year, I suggested
> this to the W3C. With his encouragement, I’m writing down the basic
> concept for this group to discuss prior to moving forward. I’m hoping
> this discussion will lead to further refinement of the concept,
> solutions to outstanding problems (and there are a few) and a better
> shared understanding of how we might put this plan into action. With
> that being said, the remainder of this (long) email is a description
> of the project. I’d appreciate your taking the time to read it and
> share your thoughts on this mailing list. 
> 
> Regards,
> 
> D-
> 
> The Spotlight Project
> 
> 1.     Overview
> 
> The Spotlight Project is intended to shine a light on the end-user
> performance of Web pages by making test results public, transparent,
> and easily understood by users. The essential concept is to establish
> a Website with current and past performance test results for the top N
> web sites and make the results publicly available to anyone in an
> easily understood manner. We would also perform continuous testing of
> Web performance under a number of different conditions, with different
> user scenarios and devices. This project would be open to
> participation and contributions from the community in a way similar to
> webplatform.org.
> 
> By making comparative test results public, using a credible,
> transparent methodology, we can call attention to current performance
> issues, provide users with a better understanding of their Web
> experience, and shine a spotlight on slow Web pages, encouraging the
> entire industry to focus on making Web surfing faster for everyone.  
> 
>  
> 
> 2.     Making Performance Data Public
> 
> The key idea behind the Spotlight Project is that performance data is
> public data. By testing popular sites and services in a public,
> transparent, and credible way, we can help focus attention on Web
> performance. Most companies with a significant Web presence have data
> on their site’s performance. The Spotlight Project aims to provide a
> comparative view of the end user’s experience for the most popular Web
> sites over time.
> 
>  
> 
> 3.     Methodology
> 
> Web testing methodologies are often controversial, and there’s no one
> ‘right’ answer on Web performance testing. However, I think that
> everyone can agree that an open, credible methodology would require
> testing along at least 4 axes:
> 
> ·       Geographical distribution 
> 
> ·       Network context   
> 
> ·       Client diversity 
> 
> ·       User scenarios
> 
> In the end the precise details of the methodology would need to be
> worked out among the participants, and like many other difficult
> topics a compromise that integrates the wisdom of many viewpoints is
> the desired result. The important points are that the methodology is
> sound, transparent, and credible.
> 
> 4.     Operational Issues
> 
> Web performance testing is not simple – that’s part of why W3C’s help
> is needed for this project to succeed. In order to make performance
> data not only consumable but appetizing to our intended audience,
> we’ll need some support. Behind the scenes, data has to be collected,
> stored, and reported to users. The associated infrastructure and
> operational aspects of the Spotlight project would be borne by the
> community at large, hopefully with donations from our contributors.
> 
>  
> 
> 5.     Why W3C?
> 
> W3C is uniquely situated to make the Spotlight project work. W3C’s
> ability to bring the major players together to support this project,
> and to lend credibility and technical insight are a key part of making
> the project successful. In addition, the value of the Spotlight
> Project’s data will increase due to W3C’s longevity as an institution
> on the Web, providing continuity that no one else can provide.
> 
>  
> 
> 6.     Long-term Evolution
> 
> The Spotlight Project is intended to make a persistent set of
> performance data available for the long term. The need for continuity
> and a longitudinal view of Web performance across different browsers,
> geographies, and sites is central to the idea of making the data
> public and transparent. Inevitably the tests performed and data
> collected, and the associated methodology, would change over time. We
> should be prepared to continuously evolve along with the Web.
> 
>  
> 
> 7.     Making the Web Faster
> 
> A big part of the Spotlight Project’s mission would be educating the
> community about Web performance. Beyond publishing data about how
> popular sites are doing, we need to provide users with the means to
> make them better. This might include guidelines, articles, forums, and
> other means of helping the community make Web pages load faster.
> 
>  
> 
> 8.     Fellow Travelers
> 
> A number of other people have had similar ideas and proposed similar
> projects. I’m indebted to them all. Multiple different aspects of the
> project proposed here have been implemented on webpagetest.org,
> HTTParchive.org, measurementlab.net, and stevesouders.com, among
> others. Akamai publishes a great report on the State of the Internet
> quarterly, and I’d like to propose something similar for this project
> as well. Recently, x509labs.com created a similar project for OCSP
> providers, publishing comparative tests in a very similar way to
> what’s proposed here, and achieved remarkable results in reducing
> response times globally – no mean achievement.  This proposal intends
> to build on all of these and more, and with W3C’s help, provide a
> spotlight on Web performance.
> 
>  
> 
>  
> 
>

Received on Wednesday, 29 May 2013 13:53:28 UTC