High-level goal (why TestGL?)

Define principles & practices to support the creation of useful and usable conformance test-suites.

Principle #1: Users of the test suite must understand what is tested, how extensively it's tested, what is not tested

They need to know whether this test suite applies to them, the extent to which they can rely on it, and where they might need to focus additional testing efforts.

This implies:

Principle #2: Users of the test suite must understand how to interpret the test results

Implementation developers need to understand where their implementation is deficient, and what they can do to fix it.

This implies:

Principle #3: Test execution must be repeatable and reproducible

If it's not, I cannot rely on the results, and I cannot compare my results with anyone else's.

This implies:

Potential components of test suite

Principle #4: Test suites must evolve over time

This implies:

Operational guidelines

Treat test development like product development - it is (or should be) a formal engineering process.

For the highest-quality test suite: