SV: [Xmlconf-developer] Updated domtest.xsd and simple attr.xml

In order to flag the conformance bugs we will have one test that will
possibly be changed, but it will remain one test after consideration.

The idea is to have a test suite that contains stable tests, not different
test suites that contain tests that are before hand known to be subject to
different interpretations. The process anticipates exactly that when stating
that dubious tests are discussed by the DOM WG which comes to a normative
decision.

Hopefully audibility will not be changed, as the tets will be kept, together
with all relevant documentation, to allow for people to check against older
version of tests that have been discussed, or what have you.

We want to end up with astable suite of tests over which there is as little
confusion as possible.

/Dimitris

-----Ursprungligt meddelande-----
Från: David Brownell [mailto:david-b@pacbell.net]
Skickat: den 23 maj 2001 19:49
Till: Dimitris Dimitriadis
Kopia: www-dom-ts@w3.org
Ämne: Re: [Xmlconf-developer] Updated domtest.xsd and simple attr.xml


> In any case I cannot see that as an argument to kepp the metadata
separate,
> if the tests change, they remain one test; passsing them through a process
> does not produce more tests or more metadata.

Actually, in the interest of reducing the number of divergent
implementations, it can be better to turn such "problematic"
tests into multiple tests.  That also lets "legacy" behaviors (such
as "the way IE4 does it", just an example) be tested for in
their own right -- sometimes they're essential conformance
points for products, though one hopes not for W3C (which
should flag all conformance bugs).

So for example, a test expecting one outcome turns from
a positive test to a negative test ... and a new test, with the
"correct" outcome, is created.  For the original test, only the
metadata changes ... unless it's being used in a profile that
tests against an alternate conformance point (not limited
to earlier versions of the W3C conformance statements).

If only metadata changes for that original test, then it's
possible for the process to include audit-based self-checks.
But if changing interpretations means changing everything
about a test ... auditability gets lost, and along with it goes
a key part of QA process integrity.

- Dave

Received on Thursday, 24 May 2001 10:27:40 UTC