Test Driven Specification Development

Jeremy J. Carroll

RDF Core WG
editor OWL Test Cases

3 March 2004

Test Driven Software Development

• test represents customer functionality
• software must conform with test
• algorithm is implementation detail

Test Driven Document Development

• test represents WG decision
• specifications must conform with test
• extensive editorial discretion enabled

WG Decisions

• issue list driven
  • issue may well have example
• might be resolved with text, (later massaged by editor into document)
• might be illustrated by test case(s)
• might just be test case

Tests and Decisions

• Test developed after decision (QAF, default, RDF, OWL)
• Test developed with text of decision (RDF, SVG during CR)
• Test is decision (RDF)
• note test need not be 'polished'

WG Decisions: Tests or Text
Pro Test

• Tests are clearer
• Tests are clearer
• Tests are clearer

• Text can be fudged (useful when needing a vote, bad later)

WG Decisions: Tests or Text
Pro Text

• Tests are particular - i.e. general text is needed
  • e.g. Test: 'a b c' is an XYZ document
  • bad text: the only XYZ document is 'a b c'
• text might be useful in decision, where generalization is unclear

Normative Tests

• If test perceived as the decision then
  • text follows test not vice versa
  • specification document follows test document not vice versa (contrast with OWL)
• high value on declarative tests
• e.g. these two documents are equivalent
• e.g. this is an XYZ document
• e.g. this is not an XYZ document

Tests and Testing

• Such normative tests may be repurposed for SW development
• sometimes it works, sometimes it doesn't
• e.g. OWL 'Extra Credit' Tests are not very appropriate for SW development (typically can be failed but too hard to pass)

Goal of Tests in W3C

• jjc: higher quality specifications
  • corner cases clarified
  • normative, declarative, examples
• QAF: perhaps, higher quality implementations of specifications
  • conformance test suites
  • specs which enable conformance test suites

Tests as Tech Report?

• e.g. OWL Test Cases - all tests in-line, huge
• WG makes decisions up to and beyond publishing Rec
• normative Tests should be normative
• Test document should be finalized after the others (perhaps six months)
• QAF says 'no'

Limitations?

• Possibly more appropriate when document being edited is more mature, but design still in question
  • e.g. RDF Core had RDF Model & Syntax (Rec)
  • SVG were more like this after CR
  • OWL got closer to this with LC and CR
• Alternatively, needs WG maturity
  • superficially, tests are a bad way of making decisions
  • scandal of particularity