Re: Additions to SVG 1.1 2nd Edition Conformance Tests

Hi Erik,

--Original Message--:
><snip/>
>* Many of these tests are so simple that they should be joined into a
>single test IMHO.

I disagree.

One of my pet hates with the current test suite is that most of the tests
test a combination of many features and require many things to be
implemented to pass a single test.

So as unit tests they are useless.

The descriptions usually describe what the test is testing but in order
to pass a large number of other features need to be implemented.

The result is that some tests only test one feature - e.g. coords-trans-02-t.svg
whilst others such as udom-svg-237-t.svg test many individual features
in one test. Comparing those 2 tests feature-wise, it's about 30x the number 
of features implemented to pass the latter versus the former.

The result is that passing one test has no significant correlation with passing
a different test so the number of passes/failures is completely lacking
correlation with the correctness of an implementation.

i.e. two implementations may pass 90% of the test suite each, whilst
one is a highly conformant implementtion, yet the other fails in many
areas.

This is a problem with the current test suite - very few tests actually test
the feature they are testing in isolation - and that has been a criticism from
other people as well...

Breaking down the test suite into simpler tests, one per feature would allow
a more meaningful indicator of implemented feature coverage rather than
the conglomerated tests we now have.

I know it's nice to have the combined succinct test suite we now have, but
it is basically useless to a new implementor starting out - there should be 
some incremental single feature tests to allow an implementor to validate
each feature as it is implemented.

Because of that, I don't agree that any of the tests are too simplistic.
Breaking tests down to individual features benefits all implementors
and I think the existing test suite suffers from an imbalance in the
weight of implementation required to pass each of them.

Best regards,
Alex

Received on Wednesday, 17 March 2010 14:36:33 UTC