Agenda Item: enhance the compliancy section?

Hello folks,

I hope we can discuss this in tomorrow's call. We need to answer
Karl what is the WG's position regarding these comments.

Karl Dubost from the Quality Assurance (QA) domain sent us 
some late feedback and a request to improve the Conformance 
Section (Section 9 in the current draft). I'm including his
message here below.

We can improve the compliancy section during the CR period. Karl's
suggestions make sense to me and I think that they would help the 
XKMS spec and that they could be done with the experience acquired
during the preparation of the interop matrix. 

The issue is to know is if the WG wants to engage itself to make
a better compliancy section and how deep this engagement will go.
If the WG feels that it'll be able to improve the Compliancy section,
we should say something in the proposed Status of Document section [1].

Karl's also right in that there is a missing compliancy section for
part-2 and no indications whether the comliancy for part-1 should
be used. The only relationship between part-1 and part-2 for this is
that the part-1 conformance section says broadly which parts of 
part-2 have to be used (e.g., Soap/1.2 is required). The actual 
details of the bindings don't have any conformance section. 
Overall part-2 is mostly informal and gives bindings "by example" 
rather than by an exact description.

I don't know if this was the intention or if more strict language and
conformance must be used.

The QA folks also invite us to send messages to their public mailing
list (www-qa-wg@w3.org) if we would like to have more help in building
test suites or improving the Compliance section or just sharing their
experiences to help us jump-starting the process. They are also
preparing a test-suite guideline that could be interesting to us
(http://www.w3.org/QA/wg/)


Cheers,

-jose

[1] http://www.w3.org/2001/XKMS/rqim.html
--------------------------------------
From: Karl Dubost <karl@w3.org>
Subject: Re: XKMS move to CR, follow up
Date: Fri, 12 Mar 2004 13:59:14 -0500

Hi Jose,

Le 12 mars 2004, =E0 11:26, Jose Kahan a =E9crit :
> N.B. Karl, you had promised to mail in some comments concerning
> compliancy.


I'm happy to see XKMS with Conformance section now. It was not the case
in the past.  That's a big progress. I have still a few comments with
regards to it. Sometimes the language is a bit confusing because you
define Conformance Keywords.

> [347] If support for a feature is specified as REQUIRED a conforming
> XKMS implementation MUST support the use of that feature in a message
> sent by another XKMS implementation.

No ambiguities. It will define a first level (let's say level A)
profile of Conformance as in
	"A conforming Level A XKMS implementation MUST implement all the
         REQUIRED feature"

> [348] If support for a feature is specified as RECOMMENDED a
> conforming XKMS implementation SHOULD support the use of that feature
> if used by another XKMS implementation.

Here there's no way to define a profile of conformance. With the
sentence above, you can NOT say.
	"A conforming Level B XKMS implementation MUST implement all the=

RECOMMENDED feature and all the REQUIRED feature"

It's difficult to define in the used wording what is meant exactly. As
a developer it means that I have to implement all MUST as MUST and all
SHOULD as SHOULD but doesn't tell me what is the profile of
Interoperability I have chosen for my implementation.

What I have qualified by "all" can be refined if you think that some
SHOULD features do not have to be implemented all together and still
guarantee interoperability.

It seems in part it' what you have defined with "*" and "+" but it add
another level of confusion. It would be better to organize profiles
of Conformance:
	Profile A: This
	Profile B: This and That
	Profile C: This, That, and This
	etc.

> [350] Some features as specified as REQUIRED* or RECOMMENDED*. This
> signifies that the condition holds if another feature is supported.
> For example an XKMS Locate service is not required to support XML
> Signature. If however XML Signature is supported the use of Exclusive
> Canonicalization MUST be supported.

I'm not sure I like the use of examples in the Conformance section. It
might be a good way to give hints for the implementers but I think it
should belong to a separate section.

For example, it would be nice to have a hints for implementers, where
you give examples, if you think the conformance section is not clear
enough.

> [352] Where a service supports a feature that is advertised as
> OPTIONAL it is recommended that the service advertise this feature by
> means of a Web Service description mechanism. For example

This lead to interoperability problems. It's fine to declare the
feature OPTIONAL but the way the sentence is written, it makes also the
support message optional too. What about?

[352] Where a service supports a feature that is advertised as OPTIONAL
it is MANDATORY that the service advertise this feature by means of a
Web Service description mechanism.

In this way, the feature might or might not be implemented but if it
is, the other application will know it, and so will not try to guess
if it's the case or not.


http://www.w3.org/2001/XKMS/Drafts/xkms2/CR-xkms-part-2.html 

In XKMS 2.0 Binding, there is no conformance section and no reference
to the conformance section of XKMS 2.0, if it's the one which should
be applied.


In http://www.w3.org/2001/XKMS/rqim.html
s/exit criteria/entrance criteria/

I like very much the fact that the Implementation report require "A
minimum of two complete implementations". I think it's very important,
because double implementations of each feature doesn't prove
interoperability but implementability.


If you have any questions with regards to my comments. Please contact
me.

Best Regards

--
Karl Dubost - http://www.w3.org/People/karl/
W3C Conformance Manager

Received on Monday, 29 March 2004 12:10:20 UTC