Issue 1041: ICS Good Practices: 1.2 A, B., C

Issue 1041, conformance is not a yes/no proposition 

It was decided to rewrite 1.2 Good Practice B: Provide an Implementation
Conformance Statement Pro-forma, to clarify that the ICS indicates that a
feature has been implemented and is not a conformance report.  

 

In rewriting 1.2GP B, I have made some changes to 1.2GP A and 1.2 GP C,
which also talk about the ICS.  The following are my suggested revisions.
In trying to be clear as to what the ICS is, I may be a bit wordy and
redundant.  I also tried to be clear regarding the use of an ICS in
combination with conformance tests - that this was a good thing but is
basically taking the ICS and adding to it. 

 

+++++++++++++++++++++++++++++++++++++++++++=

1.2 GP A: Provide the wording for conformance claims

 

Add to Form 1: an additional (as the last bullet)

ICS URI:  http://www.mycompany.com/ICS-20032902

 

 

1.2 GP B:  Provide an Implementation Conformance Statement Pro-forma

Old: What does it mean?

An Implementation Conformance Statement (ICS) provides standardized
information about the conformance of an implementation to the specification.
It is used to indicate which capabilities and optional features have been
implemented, as well as the limitations of the implementation.  An ICS
typically takes the form of a questionnaire for implementer to complete. 

This Good Practice suggests that the specification itself include an ICS
proforma.

 

NEW:

An Implementation Conformance Statement (ICS) provides information about an
implementation to a specification, by presenting in a uniform manner the
capabilities (e.g., functions, features) and options that have been
implemented as well as limitations of the implementation. An ICS pro-forma
typically takes the form of a questionnaire or checklist, to be completed
for an implementation.  It provides the implementer a way to indicate what
has been implemented. Think of it as an inventory of what has been
implemented.  Note, that a completed ICS does not indicate conformance of
the implementation.  Hence, answering 'yes' to indicate a capability is
supported does not mean that the capability has been tested and correctly
implemented (i.e., conforms).  

 

This Good Practice suggests that the specification itself include an ICS
pro-forma.  Providing this pro-forma makes it conducive to completing and
helps to ensure consistency among completed ICSs.  

 

Old: Why Care?

An ICS provides a concise view of a specification's conformance model. View
the ICS as a template, where its organization, format and content can
provide implementers and users a quick overview of the specification's
features, subdivisions of the technology, conformance requirements, etc. It
can be especially valuable as a statement of conformance, where implementers
indicate which mandatory and optional features they implement and document
the presence of extensions. Once completed by an implementer it can be used
as part of the conformance claim. Additionally, an ICS can be used to
identify the subset of a test suite that would be applicable to the
implementation to be tested: this is useful first when establishing an
interoperability report, and then when setting up a conformance testing
program.

 

NEW

An ICS pro-forma provides a concise summary of a specification, i.e., the
capabilities and options defined in the specification as well as any
subdivisions (e.g., profiles, modules) and conformance designations that are
defined. The ICS provided with the specification is blank, waiting for the
implementer to complete.  This blank ICS provides implementers and users a
quick overview of what is defined in the specification. A completed ICS not
only provides information on what has been implemented (mandatory and
optional features), but can also be used to document the presence of
extensions or any specializations that have been made.  Once completed, an
ICS provides information that can be used to facilitate the selection of
tests that would be applicable for the particular implementation.  But that
isn't all.  Although the ICS content is independent of testing, the ICS can
be augmented by associating it with conformance tests, thus making it an
essential piece in the reporting of conformance results (see techniques in
1.2 Good Practice C).

 

Related:

Optionality, see section 4.2

1.2 Good Practice C

ETSI Making Better Standards.
http://portal.etsi.org/mbs/Testing/conformance/conformance.asp#PICS
<http://portal.etsi.org/mbs/Testing/conformance/conformance.asp#PICS> 

 

Techniques:  minor change to #2

2.  Provide space for the implementer to check:  yes (to indicate the
feature is implemented), No (to indicate it is not implemented), Not
Applicable, and space for Comments. 

 

 

1.2 Good Practice C:  Require an Implementation Conformance Statement as
part of valid conformance claims.

What does it mean?  (no change) 

This simply puts together the previous two good practices.  Not only could
the specification provide an ICS pro-forma for implementers, but also it
could require it be linked from its standardized conformance claim template.


 

Why Care? 

OLD: Providing a completed ICS with the conformance claim might help
customers and users to verify easily the level of support of individual
requirements of the specifications.  It also strengthens the value of the
claim. 

 

NEW:  Providing a completed ICS with the conformance claim might help
customers and users to quickly determine what has been implemented as well
as easily verify the level of support for individual requirements of the
specifications.  Combining the ICS with a conformance test suite, can
strengthen the claim.  Specifically, the ICS augmented with links to
conformance tests, provides a very nice way to indicate not only what has
been implemented, but also, what has been implemented correctly (i.e.,
conforms to the specification). 

 

Techniques: Add:

2.  Augment the ICS by providing links to the test suite, such that each
feature has associated with it a test (or set of tests).  Explain what it
means to check Yes or No.  Specifically, does Yes/No indicate that the
feature is implemented and passes the applicable tests or does Yes/No only
indicate that the feature is implemented. In the latter case, add an
additional column, to indicate the result of executing the tests.  To avoid
confusion as to what an ICS is, we recommend, adding an additional column.  

 

Examples:

Lofton's WebCGM example: http://www.ematek.de/viewer-proforma.html
<http://www.ematek.de/viewer-proforma.html>