Re: some gaps in our tests (quick and dirty coverage check)

C. M. Sperberg-McQueen writes:

> For what it's worth, I have been working on tools to perform a simple
> measurement of the coverage of our test set.  The short version:
>
>   1 There are (predictably) no occurrences of the nonterminals 'version'
>     and 'prolog', or the choice that involves them, in any of the
>     grammars of the test set.
>
>   2 There are no hexadecimal numbers with length > 2.
>
>   3 There is only one empty set (an empty inclusion); there are no empty
>     exclusions.

A slightly cleaner coverage report based on the spec grammar of 28 May
confirms the above and appears to show some additional points worth
attention.  Some of these involve whitespace and are thus a bit
error-prone to detect (and may thus be false alarms):

  - If the nonterminal 'whitespace' is ever realized as a tab character,
    I cannot find it.

  - If the nonterminal 'whitespace' is ever realized as a carriage
    return character, I cannot find it.

  - If there are empty comments (i.e. {}) in any input grammars in the
    test set, I am not finding them.

  - There are (unsurprisingly) no insertions in the test suite.

  - My tools currently have trouble with these constructs, but I believe
    there are ten occurrences of "" as instantiations of dchar but no
    occurrences of '' as instantiations of schar.

The automated tools do not look to see whether every character in a
character set gets used (that would pose problems for a nonterminal like
cchar, defined as ~["{}"]), but manual checks show:

  - There are no grammar rules in the test set that use '=' to separate
    LHS and RHS.

  - All three marks are instantiated ('-' 306 times, '@' 106 times, '^'
    three times).

  - Tmark is instantiated 307 times as '-' and once as '^'.


Michael

-- 
C. M. Sperberg-McQueen
Black Mesa Technologies LLC
http://blackmesatech.com

Received on Monday, 30 May 2022 17:48:55 UTC