RE: SML interop test plan

This is a good start on a difficult topic to think about.  Here's a wild
question I have on lines 37-38 (in section 5):

 

For the purpose of this test plan, two implementations are said to be
interoperable if they produce identical model validation result for each
test case that tests a required feature of SML and SML-IF.

 

What is the primary object of interoperability?  Are "implementations"
said to be interoperable (only once implied in the SML-IF spec), or are
interchange models "interoperable" (which seems the dominant view in the
SML-IF spec, see section 4.5), or is an SML-IF document "interoperable"
(also in the SML-IF spec, and SML-IF "document" is close enough to
"interchange model" to make this case essential equivalent to the
immediately preceding case).  Are we defining a new sense of
"implementation interoperability" for purposes of the test plan?

 

This sentence above seems to imply that two implementations of SML model
validators can be said to be interoperable they have produce identical
model validation results...etc.  I don't believe the specification ever
defines interoperability of SML model validators except via the
interoperability of an interchanged model.  (Indeed, the SML-IF spec
explicitly that SML model validation permits "points of variability" on
which SML-IF imposes additional constraints to address.)  In other
words, interoperability for models is defined between an SML-IF producer
and SML-IF consumer.  A model is interoperable if an SML-IF model
validator assesses the interoperability of the interchange model in a
way that corresponds to the way the producer "thinks" the
interoperability should work.

 

To take an extreme example:  My SML-IF producer understands an EPR
scheme and so builds an SML-IF document with the appropriate aliases and
embedded documents to handle my EPR SML references.  But your SML-IF
validator doesn't understand these aliases (e.g., it doesn't understand
the EPR Reference Scheme I'm using), so the model is not interoperable
between the two implementations.  In a derivative sense, the
implementations are not "interoperable".  If this is an accurate picture
of the situation, then the fact that implementation interoperability is
"derivative" might be clarified in the test plan.

 

Kirk Wilson, Ph.D.

Research Staff Member, CA Labs

603 823-7146 (preferred)

Cell:  603 991-8873

 

This e-mail may contain information that is privileged or confidential.
If you are not the intended recipient of this communication, please
delete this e-mail and notify the sender immediately.

________________________________

From: public-sml-request@w3.org [mailto:public-sml-request@w3.org] On
Behalf Of Kumar Pandit
Sent: Tuesday, April 15, 2008 10:18 PM
To: Smith, Virginia (HP Software); public-sml@w3.org
Cc: Kumar Pandit
Subject: RE: SML interop test plan

 

Good suggestions for numbering. Here is the updated pdf with
section/page/line numbers. I have also attached the source word
document.

 

I agree about not needing to duplicate the IF format since we can
readily use IF itself. The support for locator element is optional. Not
all implementations may support it which makes it hard to use for
interop testing. However, the bigger problem with multiple files is that
of change management. Change to one file may affect many tests and thus
each person making a change will need to run all implementations against
all tests to ensure no breaks. This is one of the reasons why the
document recommends a single SML-IF file per test case.

 

 

From: public-sml-request@w3.org [mailto:public-sml-request@w3.org] On
Behalf Of Smith, Virginia (HP Software)
Sent: Tuesday, April 15, 2008 3:48 PM
To: public-sml@w3.org
Subject: RE: SML interop test plan

 

Nice work on the doc, Kumar. 

 

I have a question on the test-description file - 1b. It seems like you
wouldn't have to recreate the IF format - you could just use the IF
format for the test harness, probably with the locator element. So the
test harness would just have to embed the appropriate documents into a
final IF file to run the test. And 2d would still apply here.

 

I also recommend adding page number and line numbers for easy reference
- especially if you are not providing a word doc that can be marked up.

 

--

ginny

 

 

From: public-sml-request@w3.org [mailto:public-sml-request@w3.org] On
Behalf Of Kumar Pandit
Sent: Thursday, April 10, 2008 9:31 PM
To: public-sml@w3.org
Cc: Kumar Pandit
Subject: SML interop test plan

 

Here is the first draft of SML interop test plan. Please review and send
your feedback.

 

I have not included the list of tests because I need to make some
updates. I will send it later.