Bug 024: TestRequirement Terminology

This is feedback on a Last Call Working Draft:

Evaluation and Report Language (EARL) 1.0 Schema
W3C Working Draft 10 May 2011
http://www.w3.org/TR/2011/WD-EARL10-Schema-20110510/

Why does EARL have a newly invented name for a cluster of tests that
are evaluated as one combined unit? The existing nomenclature for the
same concept seems most conventionally to be "test suite", though the
term "group fixture" also seems related in being a test suite with
environmental data:

Test suites / A test suite is a set of tests that all share the same
fixture. The order of the tests shouldn't matter.
— http://en.wikipedia.org/wiki/XUnit

Group fixtures: Indicates whether a framework supports group fixtures.
Group fixtures ensure a specified environment for a whole group of
Tests
— http://en.wikipedia.org/wiki/List_of_unit_testing_frameworks

What's most peculiar about this situation is that TestRequirement
actually sounds atomic. It sounds like a single case. What's the
difference between a Requirement and a Case: is it clear that
Requirement is the plural of Case? TestCases would have made sense.
Even TestRequirements would have made some sense, but TestRequirement
for a plurality of cases makes no sense at all.

TestSuite would most closely mirror existing convention, but there may
be more consistency gained with TestCases, as long as it could be
shown to be understood. At any rate, this also exposes an issue in the
question of whether what is currently called an earl:TestRequirement
can be documented in terms of component instances of earl:TestCase,
and how a user would go about that.

This issue is not peculiar to earl:TestRequirement. There is a general
lack of prevailing test software terminology in EARL. This probably
comes, I should note, from the fact that EARL as a language ought to
have been as much about the R (Report) as the E (Evaluation). In the
previous ten years or so this seems to have been gradually lost from
the language.

Now EARL occupies, it may be argued, a strange position between
existing software testing techniques and a more general extensible
reporting technology. It may end up lacking the benefits of either if
it occupies this middle position without being innovative about it.

-- 
Sean B. Palmer, http://inamidst.com/sbp/

Received on Wednesday, 11 May 2011 16:07:38 UTC