Re: Evergreen Formal Objection handling (ESFO)

Ha.  I'd been off writing a long response to Tzviya, and now I think I
should just paste it directly here.  (Sorry Tzviya. I'd still like your
response.)

-------
So I think the big problem, as I've meditated on this, is that "Consensus"
to Jeff and Philippe is a process of RFCs, votes, and Formal Objections - a
series of "Decisions".  To me, consensus means making sure the group works
as a team, and everyone gets on the same page, and the ongoing
specification is a reflection of the group's thinking.  This should be a
cordial conversation and push for learning and agreement - by the Editors,
but I feel like the Chair should be responsible for ensuring that consensus
as well facilitating that process.

The first of those is covered in the doc:

"This is just an example, but it illustrates that for the most part -
within the rules set by the group - the Editors of the specification have
the primary role to ensure that the specification represents the current
consensus of the Working Group. This is done by informal means: regular
conversations between the editor and participants; participants raising
issues about the spec in github; editors addressing issues and pull
requests; and editors tracking implementations to ensure that the ER is
both reflective of implementation as well as sensitive to the issues raised
by the community. The Editor achieves this consensus without issuing formal
calls for consensus; instead the Editor is continually in contact with the
group to ensure that the ER document represents a consensus."  [From
https://www.w3.org/wiki/Evergreen_Standards#Roles.2C_Responsibilities.2C_and_Consensus_in_the_Evergreen_Model
]

The second - the responsibility of the chair to ensure consensus - is
explicitly not covered, since the doc says

As opposed to the REC track, the Chair does not have the responsibility to
call for consensus or assess consensus as a result of a poll of
participants. But the Chair has an important oversight role to ensure that
the group's discussions proceed according to the procedural approach
chartered for the group, are in accordance with CEPC, and is a resource for
participants who disagree with the way that the editors are documenting the
evolving consensus of the group. If a participant believes that the editor
is not behaving fairly, the chair facilitates discussion between the
participant and editor. If that does not solve the issue, the chair may
also issue informative calls for consensus, or engage in other discussions
to see whether consensus can be reached or whether the editor can adjust
their position.

Despite this important facilitation role of the chair, they generally do
not have the authority to overrule the sense of the editor of where
consensus lies during the ER phase of a specification development as long
as the editor is following the procedure of the group. However, in an
extreme case, the chair can request that the Director look into the
situation and possibly remove the editor.


This seems weird, because a role of "facilitating" discussions between
participants and the editor seems unnecessary - and* the Chair appears to
have zero actual power in this model.*  By contrast, in the Rec process,
the Editor(s) serve at the pleasure of the Chair, who serves at the
pleasure of the Director.  If the Chair is to be fangless, I would simply
remove them - as, in fact, the WHATWG Workstream policy does.  (If a
participant doesn't get satisfaction from the Editor, appeals go to the
Steering Group, the closest analog to the Director.)

However, *I note that this ES process ignores the whole issue of how
Editors are selected/appointed *(it seems implied that the Director
appoints and removes Editors?)*. * I think it would be a good idea,
personally, to leave the role of Chair as someone who "Generally stays
neutral in discussion", as per https://www.w3.org/Guide/chair/role.html.
(In fact, nearly all of that role works for me.)  In addition, of course,
they should continue to appoint and have the power to remove Editors.

I'd like to propose text that says something like

Similar to in the REC track, the Chair has a responsibility to ensure the
Group operates under consensus.  In the ES track, it will likely be less
likely to issue calls for consensus or assess consensus as a result of a
poll of participants; however, theChair has an important oversight role to
ensure that the group's discussions proceed according to the procedural
approach chartered for the group, are in accordance with CEPC, and has the
responsibility to be an impartial facilitator to decision-making when
necessary.  Finally, of course, the Chair is the arbiter to whom
participants appeal when they disagree with the way that the editors are
documenting the evolving consensus of the group. The chair can facilitate
discussion between the participant and editor, issue informative calls for
consensus, or engage in other discussions to see whether consensus can be
reached or whether the editor can adjust their position.  Ultimately, the
Chair has the authority to overrule the Editor and remove them if necessary.


On Thu, Mar 14, 2019 at 12:35 PM Michael Champion <
Michael.Champion@microsoft.com> wrote:

> > FO is to deal with the extraordinary case that an individual is quite
> sure that the consensus is wrong and wants to appeal to a higher authority
>
> >  I would like to maintain that in the Evergreen track and the question
> is to figure out where in Evergreen the Director should get involved.
>
>
>
> I very strongly disagree and expect to (ahem) formally object to an
> Evergreen process that perpetuates the notion that the Director is engaged
> or expert enough to over-ride a overwhelming preponderance of opinion among
> technical experts in a WG.  That might have been true in the late 90’s when
> the Process evolved, it is definitely not true now.
>
>
>
> I believe Chris was explaining how WHATWG works, not necessarily
> advocating the WHATWG working mode be applied wholesale to the ER track.
> Still, in practice WGs (and WHATWG workstreams)  generally do a good job of
> building rough consensus and “appeal to authority” is rare.   I agree with
> Tzvia that there needs to be some translation from WHATWG-speak to
> W3C-speak in designing an ER mechanism.  Specifically, it is going to be
> much easier to get the W3C community to buy-in to the idea that a neutral
> Chair, not the necessarily opinionated Editor, is the arbiter of WG
> consensus.
>
>
>
> If a WG can’t reach consensus on a technical matter, it should either
> leave that feature out of the standard until the Real World has given more
> guidance, or admit defeat and go back to incubation.  Maybe that’s too big
> a change for the Rec Track  just now, but it’s how I think about Evergreen
> Recommendations.
>
>
>
>
>
> *From: *"jeff@w3.org" <jeff@w3.org>
> *Date: *Thursday, March 14, 2019 at 12:19 PM
> *To: *"Siegman, Tzviya" <tsiegman@wiley.com>, Chris Wilson <
> cwilso@google.com>
> *Cc: *David Singer <singer@apple.com>, W3C Process CG <
> public-w3process@w3.org>, Philippe Le Hegaret <plh@w3.org>
> *Subject: *Re: Evergreen Formal Objection handling (ESFO)
> *Resent-From: *<public-w3process@w3.org>
> *Resent-Date: *Thursday, March 14, 2019 at 12:19 PM
>
>
>
> Tzviya,
>
> Of course WGs proceed with consensus whether on the REC track or WHATWG or
> on a proposed Evergreen track.  FO is not to deal with the normal case.  FO
> is to deal with the extraordinary case that an individual is quite sure
> that the consensus is wrong and wants to appeal to a higher authority.  I
> would like to maintain that in the Evergreen track and the question is to
> figure out where in Evergreen the Director should get involved.
>
> ·         For the REC track, a document cannot get to REC without the
> Director processing FOs first.  So it is clear.
>
> ·         For Evergreen, since ER is a continuous status and the Director
> is not continuously available - we were trying to figure out the right
> time/place to insert the Director.
>
> ·         Chris is pointing out that in WHATWG, appeals are made to the
> SG and they may override
>
> Aside from FO, consensus might also be different between REC track and
> Evergreen.  REC track has formal steps of advancement and generally WGs
> have formal CfC's that a document is ready for advancement.  So a document
> won't get endorsement by W3C without a formal CfC.  On the Evergreen track,
> there is continuous W3C endorsement of an ER, but I don't envisage daily
> CFC's in an Evergreen WG.
>
> HTH
>
> Jeff
>
> On 3/14/2019 2:56 PM, Siegman, Tzviya wrote:
>
> I am a little confused by this discussion. We seem to be going in a
> direction that takes us far away from W3C Process and intent. Chris and I
> were tasked with coming up with language about consensus, but I am truly
> puzzled about what is so different about the ES process and the REC track
> process when it comes to both consensus and FO.
>
>
>
> My impression is that the way that most REC track WGs work when they are
> in the writing phase is not dissimilar from ES. Editors have discretion to
> make changes to documents, but that writing should reflect the intent and
> consensus of the WG. If there are concerns about changes to documents, even
> merged pull requests, they are raised to the group and discussed. Pull
> Requests can be retracted. That is why we have version control.
>
>
>
> The process outlined by Chris from the WHATWG seems to ignore the concept
> of an active and functional WG with a chair. I don’t think we need to add
> the Director overriding an FO. Why make this a Director responsibility? WGs
> resolve issues like this on a regular basis today.
>
>
>
> Can’t we simply state:  Evergreen Standards are a part of the W3C Process
> and must follow the rules of Consensus [1], including resolving objections.
>
>
>
> [1] https://www.w3.org/2019/Process-20190301/#Consensus
> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.w3.org%2F2019%2FProcess-20190301%2F%23Consensus&data=02%7C01%7Cmichael.champion%40microsoft.com%7C2c38df1aff0746eb7b3208d6a8b1f585%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636881879609406815&sdata=Ra2gLrejuQcwaLjE%2FNfkK8fsrF9m6m%2B8UCKn29JuKD4%3D&reserved=0>.
>
>
>
>
>
>
> *Tzviya Siegman*
>
> Information Standards Lead
>
> Wiley
>
> 201-748-6884
>
> tsiegman@wiley.com
>
>
>
> *From:* Jeff Jaffe <jeff@w3.org> <jeff@w3.org>
> *Sent:* Thursday, March 14, 2019 2:46 PM
> *To:* Chris Wilson <cwilso@google.com> <cwilso@google.com>
> *Cc:* David Singer <singer@apple.com> <singer@apple.com>; W3C Process CG
> <public-w3process@w3.org> <public-w3process@w3.org>; Philippe Le Hegaret
> <plh@w3.org> <plh@w3.org>
> *Subject:* Re: Evergreen Formal Objection handling (ESFO)
>
>
>
> Thanks, Chris.  This helps a lot.
>
> So modeling a FO objection policy after this (which is consistent with the
> current FO policy), I suppose we only need to say that the Director can
> override an Editor's decision.  Correct?
>
> Jeff
>
> On 3/14/2019 2:11 PM, Chris Wilson wrote:
>
> In short, the WHATWG Workstream Policy (
> https://whatwg.org/workstream-policy
> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwhatwg.org%2Fworkstream-policy&data=02%7C01%7Cmichael.champion%40microsoft.com%7C2c38df1aff0746eb7b3208d6a8b1f585%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636881879609406815&sdata=GFz00TZExDX2SK6HTKdpksHUy9PXA1CwfQ4NqE0saqU%3D&reserved=0>)
> says "Editors are responsible for the technical content of their
> Workstreams", and the "Steering Group appoints and may remove the Editor
> for each Workstream".
>
>
>
> The core of responsibilities is in
> https://whatwg.org/workstream-policy#relationships-with-other-groups
> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwhatwg.org%2Fworkstream-policy%23relationships-with-other-groups&data=02%7C01%7Cmichael.champion%40microsoft.com%7C2c38df1aff0746eb7b3208d6a8b1f585%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636881879609416815&sdata=Cx9DfBgH08SEyM%2FDS0XejQ1nPXUtAa%2Bj9FAyZCcs0Lg%3D&reserved=0>
> :
>
> 1.       Editors must respond to substantive issues raised by Workstream
> Participants in their Workstreams. Editors have discretion to resolve
> issues based on available information.
>
> 2.       If a Workstream Participant is not satisfied with an issue
> resolution, they may request that the Editor revisit the issue. If not
> satisfied with an Editor's final response, Workstream Participants may
> appeal to the Steering Group.
>
> 3.       Editors may solicit input from Workstream Participants, and may
> consider and respond to comments, suggestions, and objections from
> Contributors and the public.
>
> The conflict resolution policy is just below it, in
> https://whatwg.org/workstream-policy#decision-making
> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwhatwg.org%2Fworkstream-policy%23decision-making&data=02%7C01%7Cmichael.champion%40microsoft.com%7C2c38df1aff0746eb7b3208d6a8b1f585%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636881879609416815&sdata=9weVPttQkvwWUhsp7UVsl4ZSUUh58gigmCvU7iWxMwI%3D&reserved=0>
> :
>
> 1.       Editors may commit changes to their Living Standards without
> further review, provided they are adhering to the requirements above.
>
> 2.       The Steering Group may override an Editor's decision, or remove
> an Editor.
>
> So in short: Editors are responsible for responding to all issues.  If a
> participant is unhappy with the Editor's (repeated) response to an issue,
> they should appeal to the Steering Group, which may override or remove the
> Editor.  (This would, of course, be somewhat catastrophic, so in practice,
> working with consensus approval is highly encouraged, and is the norm.)
>
>
>
> I would point out that the Working Mode of the WHATWG also has a high bar
> for what goes IN to a Living Standard, as laid out in
> https://whatwg.org/working-mode#changes
> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwhatwg.org%2Fworking-mode%23changes&data=02%7C01%7Cmichael.champion%40microsoft.com%7C2c38df1aff0746eb7b3208d6a8b1f585%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636881879609426828&sdata=DMFJ3YBKPZ7V6Y8p6P6%2BbxRIu9VpAWYgkXjeLtv4kIo%3D&reserved=0>
> :
>
>
>
>     Each normative change made to the standard needs to meet the following
> criteria:
>
> 1.       It must have support from implementers.
>
> 2.       It should have corresponding test changes, either in the form of
> new tests or modifications to existing tests.
>
> 3.       Implementations bugs must be filed for each user agent that
> fails tests. (This is each user agent that doesn’t match the proposed
> changes. If the test changes are not adequate to reveal that, but it’s
> known through other means, the tests should be improved first.)
>
> 4.       It should have been reviewed by one or more members of the
> community.
>
> 5.       Optional or implementation-defined behavior must be very well
> motivated.
>
> Another thing of note - Editors MAY (but are not required to) tag text as
> "Object Pending" or "Under Discussion", as stated in
> https://whatwg.org/workstream-policy#optional-tags
> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwhatwg.org%2Fworkstream-policy%23optional-tags&data=02%7C01%7Cmichael.champion%40microsoft.com%7C2c38df1aff0746eb7b3208d6a8b1f585%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636881879609436836&sdata=DcCGq8FRgORupM8cXdVbohp6vF1mpTULl49K6oGyIQ8%3D&reserved=0>
> .
>
>
>
> On Wed, Mar 13, 2019 at 4:01 PM Jeff Jaffe <jeff@w3.org> wrote:
>
>
>
> On 3/13/2019 6:19 PM, Chris Wilson wrote:
>
> I note that this is, in fact, a quite complex bit of Process, and wonder
> (as Mike has introduced) if we would be better served with a process more
> akin to the Living Standard process we used in the WHATWG; putting FOs into
> the document itself, although I understand the rationale, seems like an
> attack vector for those who disagree.
>
> I'm interested in learning more about "a process more akin to the LS
> process".  I don't know much about the WHATWG process.  Can you suggest
> some process-text which would characterize what you mean?
>
>
>
>
>
> I'd again suggest that the Chair should be more responsible for
> maintaining consensus.  (And yes, I have an action item to propose some
> text.  Was this moving to a repo somewhere, or should I just do this in
> email?)
>
>
>
> On Wed, Mar 13, 2019 at 12:51 PM Jeff Jaffe <jeff@w3.org> wrote:
>
> Thanks, David.  I generally agree with your direction.  Many comments on
> the minutiae.  (And shouldn't we have this discussion on github?)
>
> On 3/13/2019 2:38 PM, David Singer wrote:
>
> Hi
>
>
>
> here’s my suggestion. Replace this
>
>
>
>  • must ensure Director review of all pending formal objections before 24 months have elapsed.
>
> ISSUE-FO: what happens if the Group refuses to accept the Director's resolution? Some ideas:
>
>          • The Working Group ceases its work
>
>          • The Working Group is no longer allowed to publish an ERS
>
>          • The document header indicates that the Director disagrees with some parts of the document
>
>
>
> with
>
>
>
> If a Formal Objection is raised against an Evergreen Standard:
>
> We need a time interval that guarantees that a FO will be taken up and
> resolved within some bounded amount of time.  The current text says 24
> months, which may be too long.  What do people think?
>
> * Until it is resolved, all copies (including the working group’s working draft, and the document linked as the current ES)
>
> We don't have WDs for Evergreen.  So it suffices to say that the ER MUST
> document the FO in the header.
>
>
>  MUST document the existence of the unresolved FO in the document header, and SHOULD document it also in the body of the text near the subject material;
>
> Not clear to me why this SHOULD isn't a MUST.
>
>
> * After resolution, the current ES must either reflect the decision (the Director’s decision, or the agreement reached with the consensus of the WG and objector under which they withdraw their FO), or cease to be published; if the working draft or other documents of the WG do not reflect the decision, the FO marking MUST be retained.
>
> Instead of having all of the notes, it might be cleaner to have:
>
> Resolution:
>
> ·         If the FO is rejected, the FO documentation is removed from the
> header and the document
>
> ·         If the FO is accepted, the document MUST reflect the Director's
> decision if it is to continue as an ER.  If the Working Group does not
> agree, then options include:
>
> o    Removing the ER designation and publishing as a Note
>
> o    Reverting to an earlier version of the ER which does not have the
> objection
>
> o    Returning to a Preliminary Draft stage until a consensus can be
> found for the objection
>
> ·         If the Director, objector, and WG develop a different consensus
> approach, then that approach is put into the document and the FO
> documentation is removed
>
> Note: if the FO is rejected, the markings are removed. If the FO is upheld, and the document can be easily adjusted (e.g. removal of an ‘atomic' feature), this should be straightforward. In complex cases, the ES may need to revert to a state to which the FO does not apply, and if there is no such state, return to provisional status with no ES publication.
>
>
>
> Note: resolution can include reaching an agreement with the objector and the objector withdrawing their FO in favor of this resolution.
>
>
>
> Note: the Director’s decision can, of course, be appealed.
>
>
>
> Note: there are too many Notes here.
>
>
>
>
>
>
>
> David Singer
>
> Manager, Software Standards, Apple Inc.
>
>
>
>
>
>