W3C home > Mailing lists > Public > public-test-infra@w3.org > July to September 2016

Data on flakiness of web platform tests

From: Rick Byers <rbyers@chromium.org>
Date: Fri, 23 Sep 2016 16:05:22 +0100
Message-ID: <CAFUtAY-pd2uek_byBBgEPKjvhw8oKKf+STKo3EdtHNWi1JVrmg@mail.gmail.com>
To: Geoffrey Sneddon <g@gms-consulting.scot>
Cc: platform-predictability <platform-predictability@chromium.org>, public-test-infra@w3.org, Boaz Sender <boaz@bocoup.com>
Geoffrey was asking for data on what web-platform-tests were flaky.  I
don't have great data from blink (since we generally don't run the ones
that are flaky) but here is what I do have:

   - All flaky blink tests on Linux
   <https://test-results.appspot.com/dashboards/flakiness_dashboard.html#showFlaky=true&builder=chromium.webkit%3AWebKit%20Linux>
   - search this list for imported/wpt to find just the web-platform-test ones

   - List of web-platform-tests and csswg-tests that are blacklisted from
   our import process
   <https://cs.chromium.org/chromium/src/third_party/WebKit/LayoutTests/W3CImportExpectations?type=cs&q=file:W3CImport&sq=package:chromium&l=1>
   - could be for multiple reasons
      - There are also some listed here
      <https://cs.chromium.org/chromium/src/third_party/WebKit/LayoutTests/TestExpectations>
with
      imported/wpt paths

Other data?
Received on Friday, 23 September 2016 15:06:19 UTC

This archive was generated by hypermail 2.4.0 : Friday, 17 January 2020 17:34:12 UTC