Re: EverTeal

Florian,

Some comments.

1. We need an explainer.  Most particularly I have the following 
concern.  We would like to encourage stakeholders to utilize the new 
processes: updating, streamlining, etc. as we will now have a more agile 
process.  But with all of the options, the reader of the process 
document will only see that there is much more to read and more apparent 
complexity.  We need to have an English language description of the 
possible paths through the document and why this is helpful.

This is particularly true because the process document text often 
includes language to handle unlikely corner cases.  An explainer can 
focus on the usual case.

2. I'm not sure that Extensible is the best adjective since we use it 
for the Extensible Web.

3. In section 5.2.6 s/discression/discretion/

4. In the fifth bullet of 5.2.6 (the new one) there are three 
paragraphs.  However, only the first one seems actionable.  The other 
two seem like parenthetical claifying text and so should be treated as 
such (e.g. by using parenthesis, or starting with the word "Note).

5. 5.2.6.  Reference Drafts.  Just a note. I haven't yet understand the 
full definition of "Patent Review Draft", so I cannot tell if its use to 
replace "Reference Draft or CR" is all-encompassing.  In particularly 
I'm concerned about migration - i.e. adoption of documents in new 
charters produced after Process2020 with the new PP - where the 
Reference Drafts are covered by Process 2019 and the current PP.

6. 6.1.2.  In the second bullet after CR, there is a deleted explanatory 
phrase which says that we publish CRs to provide an exclusion under the 
PP (which previously was the LCWD).  If I understand this correctly, 
that motivation still exists; presumably at the first entry into CR 
(which is always a CRS).  It seems worthwhile to mention that in 6.1.2.

7. 6.1.2.  There is an immediately following green box which says (among 
other things) that after CR, no additional changes are expected without 
implementation and testing.  That, and some other points in that box may 
not be true with the new CR.  We may want to reswizzle the language.

8. PR.  It seemed awkward to define Proposed Extensible Rec only as the 
flavor of PR that shows up before an Extensible Rec, but not talk about 
it elsewhere in the document.  I believe the fix is that in the 
definition of Extensible REC - rather than saying that for an Extensible 
Rec that the team MUST solicit an AC review, we should instead say that 
to get to Extensible REC, after the WG decision, that the document 
becomes a Proposed Extensible Rec (which would then automatically create 
the AC review).  Similarly, I think it would be helpful in 6.7.5, while 
discussing how to get to Extensible REC, that text is added that 
describes that after the WG decision that the spec moves into the 
Proposed Extensible Rec state.  (See related comments below.)

9. PR.  The PR section starts (and this is true in the REC section as 
well) that the document has been accepted by the Director.  For 
documents that reach REC status without formal Director transition 
review (streamline), I wonder if we need to swizzle the language a bit.  
In esssence, the REC (and PR) are achieved because of the Director: the 
combination of the the W3C Process (approved by the Director), the fact 
that there was an initial transition request (approved by the Director), 
and the WG revisions (approved by the Director approved process).  But 
it is stretching the definition to say that this was actually approved 
by the Director.

10. REC.  You've deleted the statement that RF licenses apply to REC.  I 
don't see why we should delete that.  It is still true that RF licenses 
apply to REC.  Perhaps you deleted it because RF licenses will now ALSO 
apply to other specs.  In that case, I would leave the current 
statement, but perhaps add a parenthetical "... apply to RECs (as well 
as other specs for which xyz is true)."

11. Extensible REC.  I'm not sure that the first three sentences of this 
section are helpful.  As I understand it, to advance to REC requires a 
Director decision, which is not always the case for Extensible REC 
(streamline).  An alternative might be to talk less about the process, 
and more about the implications - such as "An Extensible REC is treated 
with the same importance and gravity as a REC, even though the process 
to approve an Extensible REC is somewhat different."  And of course, it 
is good that you point out that an Extensible REC = REC from the pov of 
the PP.

12. Extensible REC.  Second paragraph. s/explicitely/explicitly/.

13. Extensible REC.  See point #8 above.

14. Extensible REC.  Last sentence. "same process".  Same as what?  Unclear.

15. 6.2.3.  There is a note that says that Update request approval is 
expected to be fairly simple.  That sounds like "marketing".  It would 
be more useful to explain WHY it is expected to be fairly simple (e.g. 
if we assume that most updates will be relatively targeted rather than 
the size of the usual transition to CR).

16. 6.2.3.1.  For Streamline, I have several concerns about some details 
of the HRG approach:

  * We have not had much discussion about "the set of criteria" under
    which a review is necessary.  Accordingly, I expect that this might
    reduce to having 5 HRG reviews for each update.
  * 5 HRG reviews for each update seems like a large amount of total
    reviews, unless there are some limitations on the frequency of
    updates.  6.2.3.1 might be a good place for a Note which says that
    it is expected that each spec gets updated at most every 6 months
    (in most cases) to reduce the level of review.
  * For the TAG, I am concerned that they might neither have "a set of
    criteria", nor have the bandwidth for all of these reviews. 
    Personally, I would be comfortable if we only required a review of
    an update for 4 HRGs.  I'm hoping that in all cases, the TAG review
    for the original CR suffices.

17. 6.2.7 Contributor License Grants.  I have no intuition why we would 
make these changes for ET, but perhaps it is related to some other PP 
change.  But in general, we should be careful about making changes that 
are not strictly needed for ET.

18. I find some confusion between Sections 6.4, 6.4.1, and 6.4.2.

The general approach of Chapter 6 is to define a maturity level and end 
the section with a description of what is next.

Here 6.4 defines CR; but at the end describes what happens after a CRS.

6.4.1 defines CRS, but does not describe what happens next (it is 
already listed in 6.4).

6.4.2 is the more conventional defining CRU and what comes next.

At a minimum there is a lack of parallel structure.

19.  6.4.  In the list of next steps after CR (or CRS actually), I 
suggest we delete that PR is the expected next step; since often the 
next step is a CRU.

20. Charters and CRS.  When I got to 6.4.1/2 I think I finally 
understood that all spec can produce CR Updates and snapshots - not only 
extensible specs.  But, it is not obvious to me why we need Extensible 
RECs to be allowed by the charters, but not CRUs.

21. 6.4.1 requires an "update request" which links to 6.2.3.  It is not 
clear that we need this long section (6.2.3).  Why can't we just say 
that to publish another CRS we need to meet all requirements of CR; 
noting the couple of exceptions (e.g. the possibility of using the 
streamline process)?

22. I'm confused about the scope of Section 6.7.4.  It does not seem to 
be about Extensible Recommendations.  Which I don't really understand.  
I would think that for non Extensible RECs we should not be changing how 
we handle substantive changes.

23. For Extensible RECs, I'm not sure why we need two different 
sections, 6.7.4 and 6.7.5.  Can't we design a single process to handle 
both substantive changes as well as new features?  Indeed, the last 
paragraph of 6.7.5 combines them - why not combine them at the outset 
and simplify the process?

24. 6.7.5.  Second paragraph.  I don't understand why new features are 
not class 4 changes.

25. 6.7.5.  I am confused about Last Call Review.  Here is how I see 
it.  When we create the new Spec we need to of course ensure wide review 
and HRG review.  Today we simply state the need for review (e.g. at CR) 
- without create a specific "Last Call Review" AC review.  So this Last 
Call Review does not seem to be the "wide review" equivalent.  Perhaps 
this is intended to be the Extensible PR AC review.  But we should just 
call it that - the Extensible PR AC review!  Surely we don't need to 
send this to the AC twice. This relates to point 8 above.

Jeff

On 10/24/2019 2:45 AM, Florian Rivoal wrote:
> Hi Process CG,
>
> Fantasai and I made a first pass at at incorporating everteal into the Everblue branch of the Process document.
>
> * Preview of the result:
>    https://w3c.github.io/w3process/everblue/
>
> * Diff against current master branch:
>    https://services.w3.org/htmldiff?doc1=https%3A%2F%2Fw3c.github.io%2Fw3process%2F&doc2=https%3A%2F%2Fw3c.github.io%2Fw3process%2Feverblue
>
> Feedback very much welcome, especially in the form of specific issues raised in https://github.com/w3c/w3process/issues with label "Everblue/teal".
>
> —Florian
> PS: As with the rest of Everblue, this is currently written assuming an updated patent policy, and references to the patent policy reflect that. We're very well aware that there's much work left to do in this area, but nonetheless feel that the rest is sufficiently complete to solicit detailed feedback.
>
>> On Aug 28, 2019, at 9:32, fantasai <fantasai.lists@inkedblade.net> wrote:
>>
>> About a month ago Ralph prompted the question of an EverTeal proposal, that would merge the Evergreen and Everblue proposals into one proposal. So I've been thinking on two questions:
>>
>>   - What would EverTeal look like?
>>   - What would I want the Process to be like if I wasn't starting
>>     from the constraints of what it is today?
>>
>> It occurred to me that both of these can have the same answer.
>>
>>
>> I have a few high-level wishes for the Process:
>>
>> 1. It should enable Working Groups to maintain their /TR specification
>>     as the authoritative copy of their spec *in practice*, which means
>>     that every edit that represents what the Working Group recommends to
>>     implementers is consistently and quickly published to /TR.
>>
>>     -> This means that recommendations to implementers do not sit in an
>>        Editor's Draft for months before being published, which causes
>>        the ED to become the authoritative copy in practice. (An ED should
>>        only exist as a temporary scratch space for working out, reviewing,
>>        and approving final edits. W3C should be hosting referenced specs.)
>>
>>     -> This means that recommendations to implementers do not sit in
>>        Pull Requests for months before being published [e.g. because they
>>        are waiting to be implemented before being finalized], which results
>>        in the authoritative copy being a pile of tentative PRs, instead
>>        of a single document where the feature's definition is explicitly
>>        shared knowledge available and obvious to all readers.
>>
>> 2. It should provide some structure, the way transition request currently
>>     do, that push a Working Group to review its own work, and that can be
>>     used by staff to catch where such work is not happening well: whether
>>     because the Working Group is new and inexperienced; or the editor,
>>     however capable and well-intentioned, let things fall through the cracks;
>>     or some other reason. (I have this opinion because of my many years of
>>     experience: the transition checks almost always result in me finding
>>     things that I missed.)
>>
>>     -> This includes checking that issues are being “formally addressed”,
>>        that comments are not summarily dismissed or continuously ignored.
>>
>>     -> This includes checking that coordination with horizontal review is
>>        happening productively.
>>
>>     -> This includes ensuring that the community (including people outside
>>        the Working Group itself) have been given a meaningful opportunity
>>        to review and provide feedback.
>>
>> 3. It would be extra nice if we could satisfy #1 while also having a
>>     higher-level view of progress: a periodic publication that batches
>>     changes into coherent sets that can be synced with a call for wider
>>     review and publicity.
>>
>>
>> Current status:
>>
>> Currently, each CR publication is treated as a transition, handling the
>> requirements for #2 as well as invoking a Call for Exclusions. The work here creates significant friction to updates, making it more appropriate to batch edits into less-frequent updates. (These types of checks are also more effective if done periodically rather than continuously.) Additionally, there's significant pressure on the legal side to batch their patent review work. So while we can streamline simpler CR transitions as Everblue tries to do, the combination of these factors (particularly the legal review) means throttling CR updates at roughly twice per year. Which doesn't satisfy #1...
>>
>> On the flip side, Evergreen Recommendations try to optimize #1 at the expense of #2. A model without checkpoints might work for senior spec engineers who understand and embrace their mandate to collect feedback and process issues, and know how to do it well through their own process. But W3C invites participation from communities who are less expert: an important part of what we do as an organization is supporting those communities through the involvement of more experienced staff, community members, and horizontal review members. The Process *should* be designed to leverage these resources and provide some guide rails. Also, we already have feedback from our horizontal review groups that they would find the Evergreen model harder to work with, in particular because they are all overloaded: it's better to put the burden on the Working Group to seek horizontal review rather than on the Horizontal Review Group to track every Working Group.
>>
>>
>> Proposal:
>>
>> Decouple CR publications from CR transitions by adopting the Evergreen model of continuous publications + snapshots. Specifically,
>>
>>   A. After the first CR publication (which is a Patent Review Draft and
>>      also invokes the traditional transition process), the Working Group
>>      can update the CR essentially at will, similar to ER.
>>
>>   B. Periodically, the WG issues a CR that is also a Patent Review Draft,
>>      similar to an ER snapshot. Like the initial CR (and unlike an ER
>>      snapshot), this publication needs to go through the transition process
>>      [possibly streamlined via Everblue adjustments], to ensure that the
>>      changes have received wide review, etc.
>>
>>   - Substantive changes since the last PRD should be continuously documented.
>>
>>   - PRD should be issued every 6-12 months if there have been substantive
>>     changes, and must be issued at least every 24 months after such changes.
>>
>> Benefits of this proposal:
>>
>>   - Solves the patent-review throttling problem for CRs in Everblue by
>>     disconnecting CR updates from patent review drafts.
>>
>>   - Addresses the horizontal review concerns with Evergreen by incorporating
>>     periodic checkpoints in connection with Patent Review Draft publications.
>>
>>   - Allows WGs to make their latest CR work available on /TR continuously.
>>
>>   - Does not rely on tooling or data that we do not have, other than a slight
>>     update to the spec templates.
>>
>>   - Resolves the relationship of Evergreen and Recommendation by merging
>>     onto a single track, eliminating concerns about the potential chaos
>>     of having two distinct Process tracks and providing a clear path to REC.
>>
>>   - Bonus: There are two tracks to walk through history: a high-level chain
>>     of Patent Review Drafts, and a low-level chain of every update. This can
>>     help reviewers and anyone trying to understand the evolution of the spec.
>>
>> References:
>>   Everblue https://www.w3.org/wiki/Maintainable_Standards
>>   Evergreen https://www.w3.org/wiki/Evergreen_Standards
>>
>> ~fantasai
>>
>>
>