Proposal to introduce test suite curators

We suffer from a lack of review on tests. The average age of a PR on the test repo is currently around 370 days, or 195 days even if we only count those not marked "awaiting-submitter-response". That's way too long.

I think that's in part because nobody in particular feels responsible for this.

I do not think we can force anyone to review tests if they'd rather be doing something else, but we could work on identifying which spec lack test reviewers, and try to find volunteers.

I think each spec needs a minimum of 2 test reviewers. They can be the same as the editors, but do not have to be. I think it has to be at least 2, because if there's only one, there's nobody to review that person's tests, even though they are fairly likely to write some.

So I suggest we introduce a new role, which I'll call Test Curator (feel free to bikeshed). The primary responsibility is not to write the entire test suite, but to ensure that reviews and merges of Pull Requests get done in a timely manner. Secondary responsibilities could include: being able to identify which are of the test suite need more work (or to declare it complete) or checking if existing tests are still valid after normative changes in the spec.

I suggest that on each spec:

* we should list the "Test Curators" , next to and separately from the Editors.

* Ask Editors if they're willing to be a Test Curator as well. If they are, list them as such.

* For Every spec that has less than 2 test curators, call for volunteers

* Have the chairs periodically (monthly?) check if more people need to be appointed (in addition or instead) for specs which still have a large queue of Pull Requests. This should be take as seriously as finding new Editors for a spec when the previous/current ones have left or aren't keeping up.

Note that this is separate from the (poorly named) OWNERS file used in the web-platform-test repo, which is merely a notification mechanism, with no implied responsibility.

—Florian

Received on Friday, 3 March 2017 02:07:21 UTC