- From: C. M. Sperberg-McQueen <cmsmcq@blackmesatech.com>
- Date: Thu, 1 Jul 2021 13:40:13 -0600
- To: public-ixml@w3.org
- Cc: "C. M. Sperberg-McQueen" <cmsmcq@blackmesatech.com>
A few months ago I remember Tomos asking whether there were any ixml constructs which did not themselves occur in the ixml grammar for ixml. Pragmatically: if an implementor uses ixml.ixml as a test case and handles it correctly, will they have seen everything they need to be prepared for?
The degree to which ixml.xml covers all the constructs of ixml depends, of course, on how you define 'coverage' and 'construct'. But as a first attempt at a serious answer to the question I just went through the grammar manually, making a list of the conditions that I think need to be exhibited in positive test cases, in order to reach the grammatical equivalent of what Glenford J. Myers calls 'condition coverage'. Condition coverage (in case you have misplaced your copy of Myers just for the moment) is a white-box coverage measure saying that for every condition tested in the code, there should be test cases making that condition take on each possible value at least once. (So if you have code governed by IF A > 0 and B = 2, you need test cases for A > 0, A =< 0, B = 2, and B <> 2, which don’t all need to be separate.)
With respect to positive test cases, I take the grammatical equivalent of that coverage level to be that for each RHS of a reachable nonterminal NT, expressions of the form shown below require tests as described:
- E? (one E, no E)
- E+ (one E, more than one E)
- E* (no E, one E, more than one E)
- E; F (NT is an E; NT is an F)
= ["ab"] (NT is realized using "a"; NT is realized using "b")
= ["a" - "f"; "0" = "9"] (NT is realized using something in the range "a" - "f"; NT is realized using the range "0" - "9")
Adding descriptions of negative test cases is relatively straightforward, but I did this exercise manually, not automatically, and I did not attempt to make a list of negative cases to be covered. It produced a list of 87 conditions that should be exercised by positive test cases.
Because I worked manually, not automatically, I may well have glided over some things that should not be glided over, but for what it's worth, these are the conditions that are *not* exhibited in the grammar of 9 June 2021 as I went through it just now.
1. ixml grammar with a single rule.
7. whitespace containing a tab.
8. whitespace containing a linefeed character.
9. whitespace containing a carriage return character.
10. empty comment (i.e. "{}").
12. nested comment.
15. rule using "=" to separate LHS and RHS.
18. mark of "^" on a left- or right-hand nonterminal.
21. multiple alternatives separated by |.
24. An alternative consisting of zero terms.
48. Name containing the namestart character _.
51. Name containing a namestart character from class Lm.
51. Name containing a namestart character from class Lt.
53. Name containing a namestart character from class Lo.
55. Name containing the namefollower character -.
56. Name containing the namefollower character '.' (full stop).
57. Name containing the namefollower character middle dot.
58. Name containing the namefollower character undertie.
59. Name containing the namefollower character overtie.
61. Name containing a namefollower character from class Mn.
62. A tmark realized as '^'.
68. A double-quoted string containing a doubled double-quote character.
69. A single-quoted string containing a doubled single-quote character.
71. An occurrence of 'encoded' (a character specified in hex) with no tmark.
73. A hex character in the range 'A' - 'F' (as distinct form 'a' - 'f').
79. A character set exclusion with an explicit tmark.
81. A character set exclusion or inclusion with no members.
84. A character set with multiple members, separated by |.
The other 59 items on the list I made all have examples in the ixml grammar for ixml.
It should be straightforward to make a few test cases that cover these 28 cases, but I hope that most of them, if not all, are covered in https://github.com/cmsmcq/ixml-tests/tree/main/tests-straw/ixml
Michael
********************************************
C. M. Sperberg-McQueen
Black Mesa Technologies LLC
cmsmcq@blackmesatech.com
http://www.blackmesatech.com
********************************************
Received on Thursday, 1 July 2021 19:40:39 UTC