XINCLUDE Conformance to SpecGL from Mark Skall on 2004-12-09 (www-qa-wg@w3.org from December 2004)

From: Mark Skall <mark.skall@nist.gov>
Date: Thu, 09 Dec 2004 15:43:19 -0500
To: www-qa-wg@w3.org
Message-Id: <6.0.0.22.2.20041209152127.051fd5c0@wsxg03.nist.gov>
Below is my review of XINCLUDE with respect to SpecGL's ICS.  I reviewed 
the normative requirements as well as the good practices.

As you can tell, XINCLUDE was deficient in many areas.  The scope was not 
explicitly defined, who and what will implement the spec is not addressed, 
there is no list of normative references (although that may be because 
there are none), and extensibility is not addressed.  Many of the good 
practices, including the inclusion of a glossary, a conformance model, and 
conformance claim language, were not followed.

In two of the requirements, I think we need to tighten up what conformance 
means.  For identifying deprecated features, and to a lesser extent, for 
including normative references, when these things are not included in the 
spec, how do we know whether they were not applicable or if the spec was 
non-conformant by leaving them out when they did pertain?  Perhaps we 
should mandate that should always be a section on these, with a possible 
response as "non-applicable" in the spec?

Mark

===============================
* 13 Requirements (Normative) *
===============================

1.1.A: Include a conformance clause.
<http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#include->http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#include- 

conformance-clause-principle
YES/NO/Not Applicable: Explain why?

Yes

2.1.A: Define the scope.
<http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#define-scope->http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#define-scope- 

principle
YES/NO/Not Applicable: Explain why?

No.  Relationships to, and differences between, other documents are 
discussed.  Some of these discussions imply scope, but scope is not 
specifically addressed.

2.2.A: Identify who or what will implement the specification.
<http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#implement-principle>http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#implement-principle 

YES/NO/Not Applicable: Explain why?

No.  This is not explicitly addressed.  The classes of products (i.e., type 
of products or services) to which the specification applies are not 
identified per se.  However, in the conformance clause,  conformance for 
"element information items" and "applications" are defined.

2.3.A: Make a list of normative references.
<http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#ref-norm-principle>http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#ref-norm-principle 

YES/NO/Not Applicable: Explain why?

No (however, it may not be applicable).  A list of non-normative references 
are included so it's possible that normative references were considered and 
determined non-applicable.

3.1.A: Define the terms used in the normative parts of the
specification.
<http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#define-terms->http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#define-terms- 

principle
YES/NO/Not Applicable: Explain why?

Yes.  There is a section entitled "terminology" where terms are defined.

3.1.B: Create conformance labels for each part of the conformance model.
<http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#conf-label-principle>http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#conf-label-principle 

YES/NO/Not Applicable: Explain why?

Yes. "Markup conformance", "application conformance", and "XML information 
set conformance" are conformance labels.

3.2.A: Use a consistent style for conformance requirements and explain
how to distinguish them.
<http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#consistent-style->http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#consistent-style- 

principle
YES/NO/Not Applicable: Explain why?

Yes.  The specification says that "The key words must, must not, required, 
shall, shall not, should, should not, recommended, may, and optional in 
this specification are to be interpreted as described in 
<http://www.w3.org/TR/xinclude/#RFC2119#RFC2119>[IETF RFC 2119].]"

3.2.B: Indicate which conformance requirements are mandatory, which are
recommended and which are optional.
<http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#req-opt-conf->http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#req-opt-conf- 

principle
YES/NO/Not Applicable: Explain why?

Yes.  RFC 2119 key words are used.

4.1.B: If the technology is subdivided, then indicate which
subdivisions are mandatory for conformance.
<http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#subdivide->http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#subdivide- 

mandatory-principle
YES/NO/Not Applicable: Explain why?

Not applicable.   No subdivisions.

4.1.C: If the technology is subdivided, then address subdivision
constraints.
<http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#subdiv-constraints->http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#subdiv-constraints- 

principle
YES/NO/Not Applicable: Explain why?

Not applicable.  No subdivisions.

4.3.A: Address Extensibility.
<http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#likehood-extension->http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#likehood-extension- 

principle
YES/NO/Not Applicable: Explain why?

No.

4.4.A: Identify deprecated features.
<http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#deprecated-feature->http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#deprecated-feature- 

principle
YES/NO/Not Applicable: Explain why?

No.  However, it may not be applicable.
Note:  When deprecated features are not discussed, how do we know if this 
is non-conformance to SpecGL or if this was not applicable (and thus it did 
conform?)


4.4.B: Define how deprecated feature is handled by each class of
product.
<http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#degree-support->http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#degree-support- 

principle
YES/NO/Not Applicable: Explain why?

Again, the answer is either no or not applicable.


=====================================
* 25 Good Practices (Not normative) *
=====================================

1.1.B: Define the specification's conformance model in the conformance
clause.
<http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#conformance-model-gp>http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#conformance-model-gp 

YES/NO/Not Applicable: Explain why?

No.  No conformance model is really present.

1.1.C: Specify in the conformance clause how to distinguish normative
from informative content.
<http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#norm-informative-gp>http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#norm-informative-gp 

YES/NO/Not Applicable: Explain why?

No.

1.2.A: Provide the wording for conformance claims.
<http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#conformance-claim-gp>http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#conformance-claim-gp 

YES/NO/Not Applicable: Explain why?

No.  Conformance claims not mentioned.

1.2.B: Provide an Implementation Conformance Statement proforma.
<http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#ics-gp>http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#ics-gp 

YES/NO/Not Applicable: Explain why?

No.


1.2.C: Require an Implementation Conformance Statement as part of valid
conformance claims.
<http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#ics-claim-gp>http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#ics-claim-gp 

YES/NO/Not Applicable: Explain why?

No.

2.1.B: Provide examples, use cases, and graphics.
<http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#use-example-gp>http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#use-example-gp 

YES/NO/Not Applicable: Explain why?

Yes.  Many examples included.


2.3.B: Do systematic reviews of normative references and their
implications.
<http://w3c.test.site/TR/2004/WD-qaframe-spec-20041122/#ref-define->http://w3c.test.site/TR/2004/WD-qaframe-spec-20041122/#ref-define- 

practice
YES/NO/Not Applicable: Explain why?

Not applicable.  No normative references.


3.1.C: Define the unfamiliar terms in-line, and consolidate the
definitions in a glossary section.
<http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#define-terms->http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#define-terms- 

inline-gp
YES/NO/Not Applicable: Explain why?

No.  No glossary included.


3.1.D: Use terms already defined without changing their definition.
<http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#reuse-terms-gp>http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#reuse-terms-gp 

YES/NO/Not Applicable: Explain why?

Not applicable.

4.1.A: Create subdivisions of the technology when warranted.
<http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#subdivide-foster-gp>http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#subdivide-foster-gp 

YES/NO/Not Applicable: Explain why?

Not applicable.

4.1.D: If the technology is profiled, define rules for creating new
profiles.
<http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#rules-profiles-gp>http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#rules-profiles-gp 

YES/NO/Not Applicable: Explain why?

Not applicable.

4.2.A: Make sure there is a need for the optional feature.
<http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#need-option-gp>http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#need-option-gp 

YES/NO/Not Applicable: Explain why?

Can't tell.  This is not testable or even ascertainable.

4.2.B: Clearly identify optional features.
<http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#label-options-gp>http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#label-options-gp 

YES/NO/Not Applicable: Explain why?

Yes.

4.2.C: Indicate any limitations or constraints on optional features.
<http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#constraints-gp>http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#constraints-gp 

YES/NO/Not Applicable: Explain why?

Yes.


4.3.B: If extensibility is allowed, define an extension mechanism.
<http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#extensions->http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#extensions- 

prohibited-gp
YES/NO/Not Applicable: Explain why?

Not applicable.  Extensibility not addressed.

4.3.C: Warn implementers to create extensions that do not interfere
with conformance.
<http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#breaking->http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#breaking- 

conformance-gp
YES/NO/Not Applicable: Explain why?

Not applicable.  Extensibility not addressed.

4.3.D: Define error handling for unknown extensions.
<http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#define-error-gp>http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#define-error-gp 

YES/NO/Not Applicable: Explain why?

Not applicable.  Extensibility not addressed.

4.4.C: Explain how to avoid using a deprecated feature.
<http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#workaround-gp>http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#workaround-gp 

YES/NO/Not Applicable: Explain why?

Not applicable.  Deprecated features not addressed.

4.4.D: Identify obsolete features.
<http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#obsolete-gp>http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#obsolete-gp 

YES/NO/Not Applicable: Explain why?

No.

4.5.A: Define an error handling mechanism.
<http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#error-handling-gp>http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#error-handling-gp 

YES/NO/Not Applicable: Explain why?

No.

5.A:   Define an internal publication and review process.
<http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#practice-principle>http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#practice-principle 

YES/NO/Not Applicable: Explain why?

Don't know.

5.B:   Do a systematic and thorough review.
<http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#review-gp>http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#review-gp 

YES/NO/Not Applicable: Explain why?

Don't know.

5.C:   Write sample code or tests.
<http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#write-sample-gp>http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#write-sample-gp 

YES/NO/Not Applicable: Explain why?

Don't know.

5.D:   Write Test Assertions.
<http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#write-assertion-gp>http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#write-assertion-gp 

YES/NO/Not Applicable: Explain why?

No.

5.E:   Use formal languages and define which from prose and formal
languages has priority.
<http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#formal-language-gp>http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/#formal-language-gp 

YES/NO/Not Applicable: Explain why?

No.


****************************************************************
Mark Skall
Chief, Software Diagnostics and Conformance Testing Division
Information Technology Laboratory
National Institute of Standards and Technology (NIST)
100 Bureau Drive, Stop 8970
Gaithersburg, MD 20899-8970

Voice: 301-975-3262
Fax:   301-590-9174
Email: skall@nist.gov
****************************************************************
Received on Thursday, 9 December 2004 20:43:32 UTC