Process proposal for modifying tests

Once tests have been published, external testsuites including the
testsuites of other standards bodies may rely on them.  There is the
concern that this may lead to problems if tests are changed subsequent
to their initial publication.
 
For example, if Implementation A is certified as qualified by passing a
test which calls for the underlining of images, and Implementation B is
later certified as qualified by passing a changed version of the same
test file, but which now calls for images not to be underlined,
Implementations A and B will not interoperate wrt underlining of images.
 
One answer would be to establish a process whereby a test file would
only change by having its contents replaced with an explanation of why
the original test needed to be removed or modified, and a pointer to the
modified test(s).
 
Another possibility might be to never revise tests in dated directories,
and clearly communicate that the contents of dated dirs are frozen and
the tests in 'current' dirs are subject to change...  The problem with
this approach is that I think people will expect tests on the W3 site to
be correct, and won't in general notice whether they're in a 'current'
or dated directory.  So I prefer the first option.
 
Would this approach be workable, or how else might we mitigate this
issue?
 
Best wishes,
 
Melinda
 
________________________________

 
<http://www.hp.com/Redirect/gw/useng_welcome/logo/=http:/welcome.hp.com/
country/us/eng/welcome.htm> 

Melinda S. Grant	
Connectivity Strategist
Imaging & Printing Group

+1.541.582.3681

Melinda.Grant@hp.com <mailto:melinda.grant@hp.com>