Re: Exit Criteria (CR/PR) Interoperability report

See my comments below preceded by "IJ:".

Karl Dubost wrote:

> Often, some people in charge of the interoperability table (Exit 
> Criteria) for a specific specification come to me because they are 
> afraid to not be able to pass to PR. They want to be sure to have the 
> right information in their documents.
> 
> In fact, there are no strict rules.
> 
> Just  for the record, a document is going through
>     WD -> Last Call WD -> CR -> PR -> Rec
> 
> If we look at the Process Document
> http://www.w3.org/Consortium/Process-20010719/tr.html#RecsPR
> 
> [NotaBene: First surprising thing, there's no mention of Exit Criteria 
> in the whole process document, but Entrance Criteria. Something to fix?]


IJ: This is not a bug. The Process Document only talks about 
entrance criteria for phase transitions. You never ask the Director 
"Are we done with CR?", you ask the Director "Can I advance to PR?" 
One of the things the Director looks at is whether there is 
sufficient implementation experience for the specification. Since 
WGs can skip CR entirely with sufficient implementation experience 
(though maybe they shouldn't), then it makes more sense to talk 
about PR entrance criteria.

I know people talk about "exit criteria" rather than entrance
criteria, but that doesn't bother me. :)


> 
> in PR,
> ------------------------
> Entrance criteria. Before advancing a technical report to Proposed 
> Recommendation, the Director must be satisfied that:
> 
> [...]
> 
> 4. each feature of the technical report has been implemented. 
> Preferably, the Working Group should be able to demonstrate two 
> interoperable implementations of each feature. If the Director believes 
> that immediate Advisory Committee review is critical to the success of a 
> technical report, the Director may advance the technical report to 
> Proposed Recommendation even without adequate implementation experience. 
> In this case, the technical report status section should indicate why 
> the Director advanced the technical report directly to Proposed 
> Recommendation;
> -------------------------
> 
> 
> In CR,
> http://www.w3.org/Consortium/Process-20010719/tr.html#RecsCR
> 
> ------------------
> The Working Group is not required to show that a technical report has 
> two independent and interoperable implementations as part of a request 
> to advance to Candidate Recommendation. However, the Working Group is 
> encouraged to include a report of present and expected implementation as 
> part of the request.
> -------------------
> 
> 
> in QA Framework document
> http://www.w3.org/QA/WG/2002/framework-20020201/qaframe-ops.html
> 
> 
> "Although not explicitly stated, the W3C Process Document supports the 
> development of conformance test materials."
> 
> and
> 
> "In an effort to meet these suggestions and address the implementation 
> requirements of the Process Document, some Working Groups have included 
> the development of conformance materials as part of their CR-exit and 
> PR-entrance criteria."


IJ: Please delete "CR-exit and" from this sentence. It adds 
confusion if people think that CR-exit and PR-entrance are different 
are essentially the same, but it's easier to model using entrance 
criteria.

 
> and
> 
> "Checkpoint 1.2. In the Working Group charter, specify completion and 
> publication of test materials to be a criterion for CR-exit and 
> PR-entrance. [Priority 2]"


IJ: Same comment: please don't talk about CR-exit.


> 
> *************************
> 
> Saying all of that, there's no clear way, for people to know what they 
> should/must do to have a clear and not ambiguous report. How to present 
> their data, How to explain why it fails, how to deal with wrong tests etc.
> 
> Maybe we need a new checkpoint. Because it's one of the formal thing 
> written in the Process document but not yet clearly explained. :)


IJ: I think it's explained very clearly. Read it without thinking 
about CR exit and I trust you will find that the process holds 
together. I don't think the description is currently broken, I think 
that people are used to talking about "exit" criteria even though 
the Process Document does not.

  - Ian




-- 
Ian Jacobs (ij@w3.org)   http://www.w3.org/People/Jacobs
Tel:                     +1 718 260-9447

Received on Monday, 25 February 2002 09:34:49 UTC