Comments on Test Cases, 5 August version, part 1

[this is not sent as chair, but as working group member]

Comments on 7 August 2007 Test Cases, latest Editors Version
http://www.w3.org/2007/xmlsec/interop/xmlsig-interop-doc/testcases.html

These are general comments on the test cases document. This does not  
have detailed comments on the individual test cases (to follow).

General Comments

g1. If we can create a version with line numbers that would help with  
review. Otherwise numbering the tests would help, e.g. with table  
numbers.

g2. I think it would be much clearer if we group tests by positive  
tests and then negative tests within a category.

g3. We need to give each test a clear English title as to purpose,  
e.g. first test in 3.2.1 is "C14N11 element with xml:lang on element  
e that has children to test that xml:lang attribute is not  
propagated.". This could be part of the table title which includes  
the #, e.g. "Test Table 1: C14N11..."

g4. We should replicate all the examples in C14N11 CR as tests,  
including the new last one.

g5. Change title of document to  "Test cases for C14N11 and XML  
Signature Interoperability" to reflect that we are testing both  
C14N11 and XML Signature. Spell out terms in the title.

g6. We may wish to revisit the notation, possibly simplify. What is  
the purpose of the notation, is this tied into tool automation?

g7 For each test we need an introductory paragraph that explains the  
purpose of the test and why it is needed. For example, in  Section  
3.2.1, Test case canXML11.xmllang#2-positive, the text could be

"This test is used to verify that the XPath selection works properly  
in conjunction with an element containing an xml:lang attribute as  
well as children. The XPath expression only selects the element itself."

g8 Do we need all the explicit negative tests?
Wouldn't a mistake appear in the appropriate positive test failing?

g9 Do we need a separate set of test cases for xml:space and  
xml:lang? If not, perhaps we can define once a set of test cases with  
a placeholder for the attribute and then substitute xml:space in one  
run, and xml:lang in another.

If we can do this then we can reduce the size of this document and  
maybe reduce potential for inconsistencies in the tests.

Thus the sections would be

xml:id tests
xml:base tests
inheritable xml attribute tests (xml:lang, xml:space)
combined tests (tests with combinations of xml attributes, e.g. from  
C14N11 CR document)

Comments by section

s1. Abstract:
Change from
"This working document defines test cases for interoperability tests  
for [XMLDSIG] in the light of two areas that have suffered changes  
since its publication of XMLSig, namely: xml namespace attributes  
management in canonicalization and the encoding as strings of  
Distinguished Names in X.509 certificates.
This document also includes references to testcases already developed  
by the [XMLDSIG] working group."

to

"This draft defines interoperability tests for [C14N11] and [XMLDSIG  
2nd Edition ]. The primary change is to correctly support  
canonicalization of attributes in the XML namespace, specifically  
xml:id and xml:base. Changes to XML Signature also include other  
clarifications including the encoding of strings used in  
Distinguished Names in X.509 certificates. Where appropriate, test  
cases previously developed by the [XMLDSIG] working group are re-used."

s2. Status of this Document
Add at end "The contents of this document are draft and subject to  
change at any time. This document has no standing and should not be  
cited as a reference other than as a draft work in progress."

s3. Document History

Move to end of document. Change to indicate editor making change.  
Rely on CVS for versioning.

s4 Introduction, section 2

Change introduction text to the following:
"There will be two types of test cases. Some C14N11 test cases will  
require verifying the output from the canonicalization algorithm  
given an input and XPath expression. Other test cases will include  
verification of XML Signatures based on inputs. For those tests that  
include XML Signatures there may be positive (valid signature) and  
negative (invalid signature) tests."

s5 Section 2.1, "test cases notation"

Change title to "Test case notation" (remove s).

s6 section 2.2 codes

Change canXML11 to C14N11

s7 Section 2.3 codes

change defCanXML to defC14N

s8. Additional test case

Add test case corresponding to section 3.8 "Document Subsets and XML  
Attributes" in C14N11 document

This is a new test case, example, with input document, XPath  
expression and canonical form.

It combines xml:base and xml:id and since it is in C14N11 should be  
included in tests.

s9 Section 3.2.1, Test case canXML11.xmllang#2-positive

Not sure why we need this test in a variant for each xml attribute,  
since the purpose of the test appears to be to select the element  
itself and not include children in the output. Can we reduce the  
number of tests here?

s10 Update references
Update reference to URI to be 3986
Add reference to XML Signature, 2nd Edition

regards, Frederick

Frederick Hirsch
Nokia

Received on Thursday, 9 August 2007 14:24:30 UTC