Re: Trimming the Rec Track Process?

I think the where we differ is I think it's fine to skip Candidate Rec 
altogether if it wasn't needed, so no need to call each Last Call a 
Candidate Rec.  Candidate Rec would instead serve as something stable 
when it was going to take a long time to fulfill the formal testing 
requirements, like in html5 and css now.  So, other specs have something 
they're allowed to point to both inside and outside W3C.


On 5/21/2013 3:01 PM, Charles McCathie Nevile wrote:
> On Tue, 21 May 2013 23:06:25 +0400, Wayne Carr 
> <wayne.carr@linux.intel.com> wrote:
>
>> (Comments without having seen it.)
>>
>> There are a set of needs satisfied one way or the other by these stages.
>
> I presume you mean the existing stages in chapter 7 - FPWD, WD, LC, 
> CR, PR, REC ...

Actually, I meant with either those or what you described as your 
changes.  But what I was trying to do was step back and list 4 
requirements that I think any set of stages and reviews should support 
in some way.
>
>> 1. add to licensing obligations to those first generated at FPWD. So 
>> need this after last change that could increase essential claims.  
>> (defined in patent policy as Last Call so stuck with something that 
>> implements last call exclusion period)
>
> Yes, unless we change the patent policy (which is not impossible but 
> any reform predicated on doing it is "courageous, minister").
Right, no need to go there.
>
>> 2. need a stable reference other orgs feel comfortable pointing to in 
>> their specs.  If we're in a situation where Recommendation takes an 
>> eternity because we're waiting for test code and implementations, it 
>> can't be REC.  So maybe not implemented or tested, but high level of 
>> confidence.
>
> If it hasn't been implemented and tested, I don't have a very high 
> level of confidence that it won't need changes. If it *has*, I have a 
> reasonable level of confidence that although it probably has nasty 
> bugs, for some set of use cases it does the job.

Yes, but isn't it pretty common that implementation gets done pretty 
early these days (for at least some features).  like html 5.0 
implementation - is most of that waiting for CR or REC?   A CR can be 
pretty stable and useful for other orgs (and for W3C to be able to 
reference where they need to in other specs).  Candidate Rec seems to be 
taking that role now.
>
>> 3. need AC to approve something considered as some sort of W3C spec 
>> (and the W3C Director/Staff to weigh in too)
>
> For me the point of AC approval, reflected by the director and staff, 
> is a pretty strong consensus that we think this is really a good idea. 
> People might not back up their words with actions, or might even give 
> the lie by their actions, but the statement of making something a 
> Recommendation means something to the outside world, and they are 
> prepared to use that back on people who don't seem to mean it.
>
>> 4. need a rock solid, it's been implemented and tested final spec 
>> (for those who need to know what has been thoroughly tested in 
>> multiple implementations)
>
> We need a kind of spec that is stable to the extent of "we know that 
> this spec (possibly within these known limitations) solves these use 
> cases".
yes, I'm not trying to argue one way or the other on the living spec 
argument.  Just saying that for some uses we need a version people agree 
is done.  Like taking it to ISO IEC JTC1. Implementers/content creators 
have moved on by the time this is done, but that doesn't make it not 
worth doing - it likely isn't going away.

>
>> Sure, Last Call doesn't need to be an independent stage.  It just 
>> needs to happen before the AC approval (so they know what they're 
>> approving).
>
> Right.
>
>> I'm not sure how Candidate Rec can serve as that sort of stable spec 
>> it it happens before the Last Call exclusion period or before the AC 
>> review.
>
> But if it happens as part of the same stage, as I propose, we get
> + the Working group committing to their spec (you can go back, but it 
> is a PITA so you try to avoid that - by getting enough review and 
> pre-testing).
> + broad public review, if we're lucky. This matters, because the real 
> test of a spec isn't whether the members of the WG get it right, but 
> whether someone who doesn't even speak the language the WG worked in 
> will get it right if they implement. Having Jill random offer his 
> experience of this is a useful, but not definitive, confirmation that 
> we were looking in roughly the right part of the forest.

But then the thing called Candidate Rec doesn't have the properties that 
it has been approved by the AC and has already been through Last Call.  
It could fail either one.  e.g. at Last Call someone discloses and 
excludes some essential claim.  So you wind up with a very useful stage 
that has no name - all those reviews past and WG thinks it's done.  And 
instead you have a review period with a fancy name.  It could be 
Candidate Review and turn into a Candidate Rec when it was done and then 
I think it does what each of us wants. But, i wouldn't make it mandatory 
in any case.  Just an option for WGs that want to do it -- like they 
think tests are going to take a long time.
>
>> So I'd suggest:
>>
>> Stages:
>>
>> Candidate Rec - experimental spec status (need last call after last 
>> substantive change, need AC approval, doesn't have multiple 
>> implementations or testing)
>
> If it happens to have implementations and testing, great. Can we get 
> outsiders to do the same and confirm? If we happen to have done that, 
> we know that there is about a 2 month wait for Recommendation, which 
> is a cheap delay for good patent commitments. (Broader implementation 
> *generally* implies broad participation in the WG, and therefore good 
> patent coverage. )

If it was a 2 month delay, I'd suggest skipping Candidate Rec 
altogether.   But, it can be years and a huge amount of money needed for 
testing.

>
>> Recommendation - final, tested spec status (W3C staff determines, if 
>> any controversy, needs AC approval)
>>
>> Reviews:
>>
>> "FPWD" and "Last Call" patent exclusion periods  - Last Call can 
>> repeat - must happen after last substantive change and before 
>> Candidate Rec
>
> My proposal is that last call and candidate rec are the same step.

I was separating out a process (reviews) from a permanent state, 
Candidate Rec document (that has passed);
>
>> "AC Review" - after last Call and before Candidate Rec.  If any 
>> issues after Candidate Rec meets all criteria to advance to Rec, 
>> another AC review if AC asks for it.
>
> I suggest one review (in an ideal world) starting when LC/CR starts, 
> and running for four weeks after the WG shows the CR criteria are met 
> (which takes a minimum of 5 weeks - all this time, the patent 
> exclusion clock is running anyway so I don't think we can speed that 
> up without a change to the patent policy).

That's good if no one excludes any patents and no one makes any comments 
the AC should consider.  Last Call exclusion period is 60 days.  I'm not 
seeing a time requirement for how long CR has to take.

>> Can do FPWD -> Last Call -> AC Review -> REC  (skipping Candidate Rec 
>> if met all the criteria).
>
> Right. Which is what happens now for the theoretical problem-free spec.

I mean above to skip the Candidate Rec stage.  We can change those 
rules.  We don't have to have it if it isn't needed for a spec.

This may be just semantics, but I'd have

FPWD (120 days exclusion - because patent policy says so).

Last Call (60 days - because the patent policy says so)

Requirement for AC review after the end of substantive changes (if there 
are further changes, some number of AC reps cab call for another AC review).

Recommendation
- with list of requirements like: WG agreement to publish; draft passed 
fpwd/last call without unresolved licensing issues (after last 
substantive change); WG addressed review comments; formal objections 
considered; passed AC review as determined by Director; number of 
implementations required by Charter; testing as required by Charter.  
(no requirement to have published Candidate Rec - Last Call serves as 
notice to seriously review).

Candidate Recommendation: Same as Recommendation except for weaker 
implementation/testing requirements (so has had patent exclusion review, 
ac review, etc.  Those alternative interoperability requirements are: 
features without the number of implementations required in the Charter 
marked as "at risk", not the same level of proof of interoperability, 
but looks OK.  Charter can set proof requirements for CR.

For a draft say developed in a CG that didn't need work in the WG, the 
minimal time would be:
FPWD - 120 days for exclusion period
AC Review 4 weeks (could do that at same time as end of exclusion 
period, but cancelled if patent issue arises until after resolved).
Time for W3C staff to make sure everything satisfied - up to 4 weeks - 
could overlap exclusion period.
So minimum time a little more than 120 days. (there is no patent policy 
requirement for last call if the draft didn't change later than 90 days 
after start of FPWD).

All the stages would be:

FPWD - 120 days
Last Call - if changes since FPWD - 60 days - repeat is substantive change
AC review - 4 weeks (repeats if substantive changes or issues raised and 
AC asks for review)
Optional Candidate Rec - if WG wants to - typically for extra time to 
implement, create tests, etc.
Rec

>
> The point of collapsing LC and CR is that one is important (triggers 
> patent exclusion period) but has very low bar to entry. The other 
> triggers a level of review based on the fact that it has a hig barrier 
> to entry (but almost no other objective criterion).

Collapsing them is fine.  But then why not collapse it by not calling it 
a Candidate Spec until it passes that review.

>
> IMHO. Simplifying because where I am it is 0130...
>
> Cheers
>
>> On 5/21/2013 6:26 AM, Arthur Barstow wrote:
>>> On 5/12/13 11:34 PM, ext Charles McCathie Nevile wrote:
>>>> Hi,
>>>>
>>>> I have been working on an idea about the Rec Track Process. I'm not 
>>>> the only one.
>>>>
>>>> Roughly, I think it makes sense to collapse Candidate 
>>>> Recommendation and Last Call together. Having put this idea about, 
>>>> and listened to some responses and ideas about it I am becoming 
>>>> more convinced, so I sat down to rewrite the chapter in the Process 
>>>> that describes Rec Track as an exercise in seeing how this would 
>>>> pan out.
>>>>
>>>> It seems to work out OK. Basically to get into last call you need 
>>>> to have done your job in the working group, and should have done a 
>>>> reasonable amount of coordination with the people most likely to care.
>>>>
>>>> I clarify that the requirement for CR isn't "two implementations", 
>>>> it is "this spec will lead to independent implementations being 
>>>> highly interoperable". Two or more implementations has been used as 
>>>> if it were a proof by induction - the first people got the spec 
>>>> right, the next people got the spec right and work with the first 
>>>> lot, so the rest will too. But this is a pretty hand-wavy approach, 
>>>> which at the same time can be used to impose massive formalism on 
>>>> stuff that doesn't need it. (There are probably ways to make test 
>>>> cases for the p element that show it doesn't work entirely 
>>>> interoperably. That should be fixed, but the idea that we should 
>>>> rescind the element until it is fixed is just a bit ludicrous).
>>>>
>>>> As a sideline, I kind of "trimmed" Proposed Recommendation a bit - 
>>>> it isn't so much a status as a waypoint. The assumption is that 
>>>> Proposed Recommendations become Recommendations, but there are 
>>>> examples of that not happening, and for the ones I know I think 
>>>> with good reason. So there is still a chance for the AC to say "no, 
>>>> stop, wait!!!", just in case.
>>>>
>>>> And I re-cast the chapter so it focuses on who must/may/should not… 
>>>> do what.
>>>>
>>>> In all of that, one nice side effect is getting the size of the 
>>>> chapter down by about 50%.
>>>>
>>>> I haven't finished, but I am interested to hear people's thoughts 
>>>> on the idea.
>>>>
>>>> I've tossed an early draft of this to the AB, so they can consider 
>>>> it in time to present it to the W3C members at the June AC meeting 
>>>> if they choose to do so. Some time in the next couple of days I 
>>>> expect to have a bit cleaner document that I don't mind being found 
>>>> in archives, with clear disclaimers to reduce excuses for 
>>>> confusion, plus fewer dodgy links, and making sure I don't breach a 
>>>> license somewhere. At that point I'll happily make it generally 
>>>> available...
>>>>
>>>> If you want to see where I am up to in the meantime I'll happily 
>>>> email you a copy.
>>>
>>> I'd like to see what you have.
>>>
>>> -Thanks, AB
>>>
>>>
>>>
>>>
>>>
>>
>>
>
>

Received on Wednesday, 22 May 2013 01:29:17 UTC