Action-155: assess test coverage in the absence of WSDL testcases

Hi everyone,
For my action 155, I temporarily "hid" the WSDL-centric testcases - 6, 7, 
13, 14, 15, and 16 - and noticed the following:
1) Assertion Protocol-2032 is the only assertion that is uncovered when 
hiding the WSDL-centric testcases
2) Even when the WSDL-centric testcases are included, the following 
assertions are not covered: Protocol-2014, -2015, and -2070.
3) It will be difficult to define a valid "required" testcase for 
assertion Protocol-2070 as it would require a lookup variant other than 
"jndi" to be used, but yet the "jndi" lookup variant is the only variant 
that is required to be supported by implementations.
4) We have very little coverage of the SOAP12-JMS-In-Out MEP (only 
test0007 covers it)
5) We have no coverage of the SOAP12-JMS-In-Only MEP
6) We have more than adequate coverage (13 testcases) of the 
SOAP11-JMS-In-Out MEP
7) We have good coverage (9 testcases each) of the SOAP11-JMS-In-Out-Fault 
and SOAP12-JMS-In-Out-Fault MEPs
8) We have no test coverage of topics as destinations.
9) Testcases 9 and 10 combine some non-WSDL and WSDL function; it would be 
good to testcases similar to 9 and 10 which do not include any WSDL

Proposals:
1) Modify test0003 so that it covers the SOAP12-JMS-In-Only MEP (rather 
than SOAP11-JMS-In-Only).   That would leave us with 1 testcase covering 
each of the two "In-Only" MEPs.
2) Make copies of testcases 9 and 10, then modify the new testcases so 
that they do not include the use of WSDL.   This will give us testcases 
which cover assertion Protocol-2001 which do not require the use of WSDL.
3) Modify a few of the testcases which cover the SOAP11-JMS-In-Out (2, 4, 
5, 6, 8-16) so that they cover the SOAP12-JMS-In-Out MEP instead.
4) Identify some existing testcases (perhaps some that are changed to 
cover SOAP12-JMS-In-Out for #2 above) that cover assertion Protocol-2032.
5) Develop two new testcases similar to testcases 1 and 3 which use a 
topic as the destination - one which uses the SOAP11-JMS-In-Only MEP and 
one which uses the SOAP12-JMS-In-Only MEP.

This completes my work for Action-155

Regards,
Phil

Received on Wednesday, 7 April 2010 04:54:48 UTC