Point Events test submission/approval process on GitHub

In our last call, I promised to write up a proposal for how people 
should contribute test cases to the W3C Pointer Events working group, 
and how we will approve those tests, now that we have decided to move 
our test suite from dvcs.w3c.org/hg to GitHub.  Here's the promised 
proposal; sorry for the delay!

Our main goals are to encourage test submissions from all members of the 
web community by making the submission process easy and familiar, and to 
use these submissions to create a high-quality, well-reviewed test suite 
for our specifications.

Our tests will be stored in a directory [1] in the 
w3c/web-platform-tests repo [2].  This helps make our test suite easy to 
discover and contribute to, since it shares a location and process with 
other W3C web platform test suites.

We will adopt the submission and approval process used by the Webapps WG 
[3][4], since we don't want to make test submitters learn a different 
process for each working group.  We will document this process in the 
Pointer Events WG wiki, and link to the documentation from the Pointer 
Events specification.  The Test Management Task Force [5] is working on 
moving this process documentation to a central location [6].  We will 
provide help and feedback on that effort as needed, and change our 
documentation to point to the new centralized documentation once it's ready.

Quick summary of the process:
1) Anyone may submit a test by creating a pull request for the GitHub repo.
2) The submitter may email the working group to announce or discuss the 
submission.
3) Any member of the working group may review the test and note any 
issues, or their approval, by commenting on it via GitHub.
4) A Test Facilitator from the WG will merge the pull request once the 
reviewers are satisfied that there are no remaining issues to be addressed.

One practical concern I have is that it will be hard to filter out 
Pointer Events test submissions from all the other submissions to the 
same repo.  To make it easier for reviewers to be notified of 
submissions without "watching" the whole repository, I suggest that we 
tweak this process slightly by making step 2 mandatory rather than 
optional.  (Or, we could talk to the Test Management TF about whether 
it's better to split the web platform test repo into multiple repos.)

If the group accepts this process, I'll start migrating our existing 
test submissions to GitHub by filing a pull request for each one (or by 
getting the originally submitters to file the pull requests). I'll also 
work with Dave Methvin to migrate the submissions from his Test The Web 
Forward repo [7].

[1] https://github.com/w3c/web-platform-tests/tree/master/pointerevents
[2] https://github.com/w3c/web-platform-tests/
[3] http://testthewebforward.org/resources/github_test_submission.html
[4] http://www.w3.org/wiki/Webapps/Testing
[5] http://www.w3.org/wiki/Testing/Test_Management_TF
[6] 
http://lists.w3.org/Archives/Public/public-test-infra/2013AprJun/0066.html
[7] https://github.com/dmethvin/pointerevents-test/

Received on Tuesday, 4 June 2013 08:35:36 UTC