Re: [External] : Re: Proposal: Asynchronous review process for specification changes

Hi Ben,

you started a brief conversation about this problem at the end of today's architecture call,
and I share your frustration to a great extent.

If I just rethink how today's call went:
- some participants are coming 10-15 mins late
- some proposals are too new and participants did not have time to review
- a realistic lead time of several days to do pre-discussion in the issue would help -  so only the remaining open points can be clarified and decided on
- We spent quite some time to understand the implications of the new discovery section - this has not been a topic for the profile so far and I think the time was necessary

I agree that we could speed up on editorial changes, e.g. your cleanup PR, which only deletes obsolete comments, is ready to merge right now.
Currently it has a merge conflict, please resolve it and mark it with the "editorial" label.

Your super large MR from several months ago did not get consensus and was causing a lot of discussions because it was touching too many contentious areas.
Breaking it down into pieces was the right decision and I'm positive that your smaller MRs will get reviewed and integrated much faster.

I'm sure other TFs in other WGs have the same problem and hope to hear from other TF leads / team contacts on how these problems are solved somewhere else.

Kind regards,

Michael


On 11. Nov 2021, at 18:24, Ben Francis <ben@krellian.com<mailto:ben@krellian.com>> wrote:

Hi all,

This is just a gentle nudge to say that I haven't yet received any feedback on this proposal, though other working group members have expressed an interest in adopting a process along these lines.

I'd be really grateful for any feedback, positive or negative, since I am still struggling to get timely reviews on specification contributions and deadlines are looming!

I feel sure we could make more progress if we made more effective use of the tools available to us.

Many thanks

Ben


On Tue, 3 Aug 2021 at 16:27, Ben Francis <ben@krellian.com<mailto:ben@krellian.com>> wrote:
Dear members of the WoT Working Group,

At the end of the WoT Architecture call last week Michael McCool and I had a brief conversation about the approval process for landing pull requests on W3C WoT specifications. I suggested that there might be an opportunity to make the review process more efficient by using asynchronous tools, rather than the current approach of only merging pull requests following a resolution in a synchronous weekly conference call.

Currently:

  1.  It is common to get to the end of a two hour meeting without having managed to review all of the open pull requests in a given repository
  2.  If anyone is missing from the call who may have feedback, the discussion is often deferred for a week or more until they are available to join a call
  3.  When pull requests are reviewed there isn't always a clear resolution about whether or not to merge and discussion can end up being paused until the following week
  4.  Sometimes only small changes are needed to a pull request before landing it, but it has to wait another week for a chance for another review

Based on experience from other W3C Working Groups I'd therefore like to propose using GitHub's in-built code review tools to formally review pull requests without having to wait for the stars to align in a weekly call.

It seems to me that this form of asynchronous decision making would also be more in line with the Working Group's existing Decision Policy<https://urldefense.com/v3/__https://www.w3.org/2019/10/wot-ig-2019.html*decisions__;Iw!!ACWV5N9M2RV99hQ!aVfkWD9j53QA2swEfBp4toJ_ffLneIHsK4s1FKhcPnOKHQE16ytI10bKfI8rpnU2XSE5$>.

Non-normative Changes


  *   For non-normative or editorial changes to the wording of a specification, I propose that a code review approval from a single Editor of the specification should be sufficient to merge a pull request.reating

Normative Changes

  *   For normative or breaking changes to a specification, I propose that an approval be required by all active Editors of a specification. Editors should use GitHub's code review tool to either:
     *   Approve the pull request (approval may be conditional on making some small changes like fixing typos before landing, noted in a comment)
     *   Request changes to the pull request, explained in a comment
     *   Comment to say that they have no strong opinion, deferring to the opinion of other editors
  *   A pull request can be landed once all editors have either provided their approval or deferred to the other editors
  *   Editors are expected to provide formal reviews, but all members (and non-members) may contribute to the public discussion on GitHub
  *   If consensus can not be reached asynchronously, then a synchronous discussion in a web conference may be needed to arrive at a consensus. If a unanimous decision can not be reached then Chairs may call for a group vote to resolve a deadlock in line with the Working Group's existing Decision Policy<https://urldefense.com/v3/__https://www.w3.org/2019/10/wot-ig-2019.html*decisions__;Iw!!ACWV5N9M2RV99hQ!aVfkWD9j53QA2swEfBp4toJ_ffLneIHsK4s1FKhcPnOKHQE16ytI10bKfI8rpnU2XSE5$>
  *   If there are Editors of a specification who are not currently regularly available for reviews and are not able to consistently review changes within a week or so, I propose that they should temporarily or permanently be moved to an "inactive" or "emeritus" editors list so that they don't block the review process

Note that the beauty of version control is that decisions need not be final. It is easy to revert a change or modify it with a follow-up commit. There are also plenty of opportunities to provide feedback on changes after they have landed, through the various stages of W3C publication.

I further suggest this process could be aided by:

  1.  Descriptive commit messages and pull request titles, ideally referencing an issue number that a patch relates to
  2.  Configuring a simple template for pull requests where authors can select whether they consider the change to be normative/breaking or non-normative/editorial and therefore which level of review they are requesting

I would value your feedback on this proposal, which could potentially be discussed in tomorrow's main WoT call.

It is also my hope that more efficient asynchronous working practices may eventually allow for a drastic reduction in the number of W3C WoT meetings conducted each week, which has in recent months ballooned out of control to around twelve hours a week.

Best regards

Ben

--
Ben Francis
Founder
Krellian Ltd.<https://urldefense.com/v3/__https://krellian.com__;!!ACWV5N9M2RV99hQ!aVfkWD9j53QA2swEfBp4toJ_ffLneIHsK4s1FKhcPnOKHQE16ytI10bKfI8rpiC0_Hl4$>

[https://docs.google.com/uc?export=download&id=1e-zfQW-hY6iJvAt3qWB3xWhUwH-_tkru&revid=0B27LYskeHykMamdnTE1BYmp0SUNDRzRVbFlxTFFxM2xvNzdJPQ]


--
Ben Francis
Founder
Krellian Ltd.<https://urldefense.com/v3/__https://krellian.com__;!!ACWV5N9M2RV99hQ!aVfkWD9j53QA2swEfBp4toJ_ffLneIHsK4s1FKhcPnOKHQE16ytI10bKfI8rpiC0_Hl4$>

[https://docs.google.com/uc?export=download&id=1e-zfQW-hY6iJvAt3qWB3xWhUwH-_tkru&revid=0B27LYskeHykMamdnTE1BYmp0SUNDRzRVbFlxTFFxM2xvNzdJPQ]