Re: review policy and Sapporo discussion minutes

On Fri, Jan 29, 2016 at 7:15 AM, Nigel Megitt <nigel.megitt@bbc.co.uk>
wrote:

> All,
>
> Following the topics raised by Glenn in yesterday's meeting I looked over
> the minutes from Sapporo at
> https://www.w3.org/2015/10/28-tt-minutes.html#item05 and found:
>
> [[
> plh: You need to think about who can merge the PRs. Just the Editor?
> glenn: I propose that the Editor(s) of the document merge the PRs and
> coordinate between themselves.
> ]]
>
> and slightly further down:
>
>
> [[
> nigel: How do we know what's reasonable in terms of review time for
> example?
> ... I think if people don't Watch then they shouldn't expect to be told
> about PRs to review.
> plh: You have weekly telecons so you can discuss it there. Or editors can
> accept non-controversial changes.
> zcorpan: For typo fixes the editor can just commit to master without a PR,
> if it doesn't need discussion.
> ... For an external contribution, then the Editor can merge directly
> without consulting the group.
> ... We leave it up to the Editor to make the judgement call for if a PR
> needs group review.
> ... If so then we should allow a week or whatever.
> nigel: The unstated thing here is that we would also link the ED to
> echidna to auto-publish
> ... to /TR. Right now publishing to /TR is subject of a group review. This
> way if only the
> ... Editor chooses what is significant that has kind of taken something
> away from the group.
> ... So I think anyone should be able to put their hand up and say that
> they want a bigger review.
> pal: I think the risk is that people get snowed under by emails from
> github.
> zcorpan: We can have a rule that if the Editor notices that a change is
> significant then
> ... they email the reflector to say this PR (linked) is a major change.
> plh: That's the safest way to start, then if you get more confident then
> you can change
> ... away from that if you want to.
> ]]
>
>
> There were no contrary views expressed.
>
> We also resolved:
>
> [[
> RESOLUTION: When we are on github using the PR mechanism then we
> auto-publish WDs to /TR every time there's a new commit on the relevant
> branch.
> ]]
>
> To my mind this means that we did discuss at least in part the issues that
> Glenn raised yesterday and came to a conclusion on them.
> * We did not record as an explicit Resolution that Editors should merge
> into the ED main branch; nevertheless, it seems that all the Editors are
> happy to take on this role.
> * We did agree that a 2 week review process is not required for every
> change, at the Editor's discretion, which addresses one of Glenn's
> concerns if I understood correctly.
> - Of course if anyone does wish to explicitly request a review before
> publication they have the option to do so, and should do as soon as
> possible.
>
> Considering the points about Review Then Commit or Commit Then Review, I
> agree with Glenn that we did not discuss changing our policy, however I do
> not believe such a policy change was implied, merely a change in where and
> when the work happens. Otherwise I also would have been uncomfortable with
> the above Resolution.
>
> To explain: whereas we used to Commit Then Review on Editor's Drafts and
> Review Then Commit on Working Drafts, since we have resolved to auto
> publish WDs based on EDs the Review stage is now necessary prior to
> committing to the ED master branch.


During the discussion and resolution, I'm pretty sure I mentioned that I
was not (necessarily) planning to use the new auto publish mechanism for
TTML2, and that I reserved the right to continue to use the existing
(pre-github) process even having transitioned the repo to github.

I'm not sure that got minuted or not, but that was certainly my take away.
In particular, I was happy to accept the new auto publishing process for
IMSC and perhaps other new work, but I was not ready to make that
transition for TTML2.


> Pull Requests facilitate this review
> process. Effectively for each proposed change we're creating a separate
> mini ED in the form of a new branch on which we can Commit Then Review to
> our hearts' content, even after making a Pull Request from it (the PR is
> not a snapshot but tracks any changes made to its source branch until the
> PR is merged or deleted). When everyone is happy the Editor merges it and
> it gets reflected in the WD according to the automated process.
>

I am happy to use PRs with TTML2 provided the Editor has full discretion on
when to perform a merge into master. At the same time, I trust that the
Editor will use common prudence to determine of a wider or deeper review is
needed before a merge. In particular, I don't want the WG to impose an
inflexible/arbitrary set of rules on the Editor in making this
determination; however, guidance to the editor is fine.


>
>
> The benefit this gives is that the WD is always up to date with changes
> for which there is consensus, and we do not find ourselves needing to tell
> external people interested in the spec 'here's the official WD URL but you
> need to click through to the ED to see the latest changes' because the WD
> is x months out of date. Out of date WDs (and CRs but that’s another
> conversation) have caused problems in the past and do not help the
> progress of our specifications. We are also recommended not to allow WDs
> to get out of date (see below).
>
> Looking at the Process at
> https://www.w3.org/2015/Process-20150901/#WGChairReopen the Chair is
> permitted to reopen a decision in one of three scenarios, none of which
> applies here.
>
>
> I believe no change is needed from the working model and resolutions
> recorded in Sapporo.
>
>
> This topic was raised chiefly in the context of TTML2. As a related aside
> I note that the Process at
> https://www.w3.org/2015/Process-20150901/#revised-wd makes
> recommendations:
>
> [[
> A Working Group should publish a Working Draft to the W3C Technical
> Reports page when there have been significant changes to the previous
> published document that would benefit from review beyond the Working Group.
>
> If 6 months elapse without significant changes to a specification a
> Working Group should publish a revised Working Draft, whose status section
> should indicate reasons for the lack of change.
>
> ]]
>
> We're not meeting either of those recommendations right now for TTML2,
> last published as WD (FPWD) on 2015-02-12, almost a year ago. Actioning
> our resolution to auto-publish would certainly help us to meet them.
>

First, I agree that it is time for a new WD for TTML2, if for no other
reason, to meet the heart beat requirement.

Regarding the first of these, do you believe there are outside reviewers
that we believe would provide input if we created WDs more often (as
opposed to EDs)? My experience is those that care are monitoring the EDs
already.


>
> Kind regards,
>
> Nigel
>
>