Test Review Process Revision

HP has been test-driving a wiki-based test submission system, and
one thing that came out of that discussion is that the review process
seems to need some tweaking. Melinda and I have been considering the
following proposal.

The current process is

   1. Contributor submits a test.
      -> Submitted
   2. Someone who doesn't work for the same company OR an
      owner or peer reviews it, and if it passes review,
      approves it.
      -> Approved
   3. Someone (usually an owner or peer) checks it in.
      -> CheckedIn

The proposal is

   1. Contributor submits test.
      -> Submitted
   2. Anyone reviews and, if it passes review, accepts it.
      -> Accepted
   3. Owner or Peer either accepts reviewer's judgement (if
      known to be competent) or reviews the test himself.
      -> Approved
   4. Someone (usually the owner/peer in #3) checks the test in.
      -> CheckedIn
where steps #2 and #3 can be collapsed if an Owner/Peer is reviewing.

The change has two effects:
   a) It requires all tests to pass review by someone known to be
      competent rather than just anyone. (However this person can
      still be someone other than an Owner or Peer.)
   b) It encourages tests to be pre-screened by other contributors
      (who could be employees of the contributor's company) before
      hitting the bottleneck (Owner/Peers). This will hopefully also
      have the side effect of training more competent reviewers.

Some technicalities:
   - The checkin comment should name the peer reviewer: r=name
     if s/he reviewed the test, rs=name if s/he relied on the
     previous reviewers' judgement.
   - For CSS2.1 we have a "Junior Peer": under this process he
     would qualify to perform step #3 subject to the current
     same-employer restrictions.

Comments?

~fantasai

Received on Thursday, 4 September 2008 06:38:20 UTC