RE: QA WG Review of the XML Protocol SOAP 1.2 specification

-- just noticed that these were not also directed to xmlp-comments

-- another email contains a clarification on the original comments:
===============================
All,

Let me add a small clarification to Kirill's message, that may save some
wondering and head scratching.

At 01:48 AM 7/29/02 -0700, Kirill Gavrylyuk wrote:
      [...]
      Attached you may find the analysis document for SOAP 1.2
      specification against the checkpoints from the draft of the QA
      Specification Guidelines [5].
      [...]

      [5] http://www.w3.org/QA/WG/qaframe-spec

The referenced document versions at [5] correspond to our last /TR/
publication.  For the past few weeks, we (QAWG) have been implementing a
major reorganization of the document, and we have been moving through a
series of Editors drafts, on the path to pending new WG versions and a new
/TR/ publication.

Kirill's review is based on the 7/15 Editors draft,

http://www.w3.org/QA/WG/2002/07/qaframe-spec-0715

This correct document version is also linked from the head of his review.
Again, it is very different from [5], so beware and do not use that
reference.

Regards,
Lofton.
======================================

............................................
David C. Fallside, IBM
Ext Ph: 530.477.7169
Int  Ph: 544.9665
fallside@us.ibm.com

----- Forwarded by David Fallside/Santa Teresa/IBM on 08/06/2002 12:37 PM
-----
|---------+---------------------------->
|         |           "Kirill          |
|         |           Gavrylyuk"       |
|         |           <kirillg@microsof|
|         |           t.com>           |
|         |                            |
|         |           07/29/2002 01:48 |
|         |           AM               |
|         |                            |
|---------+---------------------------->
  >-----------------------------------------------------------------------------------------------------------------------------|
  |                                                                                                                             |
  |       To:       David Fallside/Santa Teresa/IBM@IBMUS                                                                       |
  |       cc:       <lofton@rockynet.com>, <karl@w3.org>, <qa-chairs@w3.org>                                                    |
  |       Subject:  RE: QA WG Review of the XML Protocol SOAP 1.2 specification                                                 |
  |                                                                                                                             |
  |                                                                                                                             |
  >-----------------------------------------------------------------------------------------------------------------------------|



Hi David,

Here are the detailed results of my review of the 4 documents [1-4] on
behalf of the QA WG. I apologize for sending them late for the reasons
stated previously. Hope the issues raised below will be still considered.

Attached you may find the analysis document for SOAP 1.2 specification
against the checkpoints from the draft of the QA Specification Guidelines
[5]. Note the "Not satisfied" checkpoints, issues are summarized below.

As I noted in my previous message, overall I find the SOAP 1.2
specification parts [1-3] to be in a good condition from testability point
of view, especially coupled with the "Assertions and Test Collection" [4].

Nevertheless, there are several issues that I find to be critical, and that
could be easily fixed:

      1.    There is no dedicated Scope section that would explain what is
      in scope and what is explicitly left out of scope of the
      specification.
      2.    There is no dedicated Conformance section that would
            o       Define what is the object of the spec (SOAP Processor)
            and what is it.
            o       when an implementation could claim conformance to the
            SOAP 1.2 spec, and what does it mean.
            o       clearly state that Part I is obligatory and any adjunct
            from the Part II is optional. What combinations of the adjuncts
            in Part II are allowed.
            o       State explicitly, does the implementation of the Part I
            that does not use any of the adjunct of the Part II still
            conform to the SOAP 1.2 specification.
      3.    Embedded in the issue 2. Not clear if the implementation is
      required to implement any of the adjuncts from the Part 2 in order to
      conform to the SOAP 1.2 specification.
      4.    Embedded in the issue 2. Not defined what can be called a "SOAP
      Processor".
      5.    For some of the multiple-choice assertions, it is not
      explicitly defined whether the choice must be consistent by the SOAP
      Node or not.
      For example, in the section 2.4, assertion regarding mustUnderstand
      SOAP headers that allows to either process the Header marked as
      MustUnderstand or generate a Fault message. It is not clear under
      which circumstances the behavior of the SOAP Node MUST remain
      consistent.

Let me know if you have any questions. I'll be back from vacation August 6
(superscript: th), checking email infrequently. In my absence, Lofton could
answer your questions regarding the QA Specification Guidelines.

[1] http://www.w3.org/TR/2002/WD-soap12-part0-20020626/
[2] http://www.w3.org/TR/2002/WD-soap12-part1-20020626/
[3] http://www.w3.org/TR/2002/WD-soap12-part2-20020626/
[4] http://www.w3.org/TR/2002/WD-soap12-testcollection-20020626/
[5] http://www.w3.org/QA/WG/qaframe-spec



> -----Original Message-----
> From: Kirill Gavrylyuk
> Sent: Sunday, July 21, 2002 6:03 PM
> To: 'fallside@us.ibm.com'
> Cc: 'lofton@rockynet.com'; 'karl@w3.org'
> Subject: QA WG Review of the XML Protocol specifications
>
> Hi, David,
> As you probably know, I volunteered to do a review for recently published
> XMLP specifications on behalf of QA WG.
>
> Unfortunately due to technical reasons I cannot send it to you in time
> (I'm on vacation and lost access to the review materials that I prepared
> on my desktop in Redmond).
>
> Overall the core parts in combination with the test document look good
> from testability standpoint. Detailed comments/suggestions will be sent
> later next week (Friday, 7/26th), upon my return to Redmond.
>
> Kirill
 (See attached file: soap12-qa-review.zip)