- From: jugglinmike <web-platform-tests-notifications@w3.org>
- Date: Fri, 31 Mar 2017 19:59:30 GMT
- To: public-web-platform-tests-notifications@w3.org
Previously, the "stability checker" script would report results for
duplicate test names in the same format as results for non-duplicate test
names, without any additional commentary. For example, a test run for a
test that included two stable subtests with the same name ("`test #1`") and
one unstable subtest ("`test #2`") might produce the following output:
| Test | Subtest | Results | Messages |
|--------------------|-----------|----------------------------|----------------------------------------|
| `/infra/demo.html` | `test #1` | **PASS: 20/10** | |
| `/infra/demo.html` | `test #2` | **FAIL: 5/10, PASS: 5/10** | `assert_true: expected true got false` |
Without proper context, these results caused confusion for contributors.
Extend the rendered output to communicate the reason for failure more
directly by including the count of tests in excess, e.g.:
| Test | Subtest | Results | Messages |
|--------------------|-----------|----------------------------|----------------------------------------|
| `/infra/demo.html` | `test #1` | **PASS: 20/10**, EXCESS:10 | |
| `/infra/demo.html` | `test #2` | **FAIL: 5/10, PASS: 5/10** | `assert_true: expected true got false` |
View on GitHub: https://github.com/w3c/web-platform-tests/pull/5305
Received on Friday, 31 March 2017 19:59:42 UTC