- From: Philip Jägenstedt <foolip@google.com>
- Date: Wed, 17 May 2017 14:29:33 +0000
- To: Bob Holt <bob@bocoup.com>, Jeff Carpenter <jeffcarp@google.com>
- Cc: public-test-infra@w3.org
- Message-ID: <CAARdPYdxaCE4z6AX1gLu-B25h=8Mum4LOz92yHm5v3uDnf0Fkg@mail.gmail.com>
I hadn't thought clearly about this overlap, but it's a good point, Jeff. For the dashboard it's not terribly important that results come in fast, one a day or so should suffice. For PRs, on the other hand, it can't really be too fast, you could imagine pushing and getting results immediately as a way of iterating on a test. If the infrastructure for running the whole test suites were also flexible enough to run individual tests on-demand and with response times no worse than we currently get on PRs, then that sounds pretty promising indeed. There is one thing that a PR tool would do that the dashboard does not, and that is to compare to the results before the changes. On Wed, May 17, 2017 at 2:10 PM Bob Holt <bob@bocoup.com> wrote: > I was hoping someone else would get to this before me... > > I don't have a strong preference in terms of working on this in a silo > versus pitching in on the dashboard. > > I will say that it looks like the WPT Dashboard is a view into a complete > daily run of all web platform tests. The tool described here deals with > aggregating the results of discrete (perhaps multiple) runs of specific > tests tied to PRs on w3c/web-platform-tests, and commenting on those PRs. I > think the question comes down to whether those scopes appear to overlap or > not. I have thoughts which you can probably suss out given how I framed the > question, but I don't think it's my call to make. > > If these do end up separate applications, I can imagine sharing UI > components between the two, if nothing else but to give a consistent > experience. > > On Tue, May 16, 2017 at 2:19 PM, Jeff Carpenter <jeffcarp@google.com> > wrote: > >> I certainly don't want to step on any toes here, but I wanted to throw my >> opinion out there since I see possible opportunities for saving time and >> effort. From my perspective it looks like the project I'm working on, WPT >> Dashboard, and this project have very similar aims: >> >> - WPT Results Consolidator: Collect and display WPT results >> - WPT Dashboard: Run, collect and display WPT results >> >> I can't help but wonder if it would save time to build this on the same >> infrastructure. Adding a couple of endpoints to collect and display WPT >> results by revision would be a fairly simple addition to the dashboard. The >> proposals, as planned, would mean we'd both be building a UI to display WPT >> test results - this sounds like the biggest duplication of efforts. >> Additionally the dashboard, being an App Engine app, also has the benefit >> of not needing server setup or maintenance, or the potential to run out of >> disk space. (see design doc here: bit.ly/wptdashboard-design-doc) >> >> On the other hand I completely understand there are reasons why people >> might want these to be separate. I would love to hear your thoughts on >> this. If this sounds like something worth pursuing further, I'm happy to >> elaborate more specifically on how this would work technically. >> >> On Tue, May 9, 2017 at 6:33 AM Philip Jägenstedt <foolip@google.com> >> wrote: >> >>> Very much looking forward to fewer notifications in my inbox, thanks for >>> putting this together, Bob! >>> >>> >>> On Tue, May 9, 2017 at 1:49 PM Bob Holt <bob@bocoup.com> wrote: >>> >>>> Hi all, >>>> >>>> There has been some discussion of late about the amount of commenting >>>> done on pull requests in the w3c/web-platform-tests repository. While >>>> various people find various comments useful, the sum of commenting can be >>>> distracting for owners and contributors. >>>> >>>> After some brief discussion with James Graham and others, we're >>>> proposing a web application that will be responsible for consolidating all >>>> of the information currently appearing as bot comments and boiling that >>>> information down to a single comment that it updates as results change. >>>> >>>> I have started a short specification document here: >>>> https://docs.google.com/document/d/1HK5roexqU2Nd2iTMKO4UM5Gc3b5vuFm06J3E06iGcxM/edit?usp=sharing >>>> >>>> It should be open for public comment via that link, I am in the process >>>> of scaffolding out the application now. I will update this document >>>> periodically as I receive comment and make progress in the scaffolding, >>>> >>>> Thanks, >>>> Bob >>>> >>> >
Received on Wednesday, 17 May 2017 14:30:17 UTC