moving the test suite?

Discussion and WG action requested on the following.

I've inquired with the W3C systems group, and they remain unwilling
to give consent to any plan for the XSD test suite that would involve
checking the individual test files into CVS and making them and their
history available on the Web through cvsweb (as is now done for the
metadata and some but not all test cases in XSDTS, on the machine
dev.w3.org).

They are happy, however, for us to move the test suite to test.w3.org,
and allow the individual test files to be browsed and served from there,
provided we use Mercurial (hg) to maintain it, rather than CVS.

This seems to entail a certain amount of work and planning on our 
side; I'm sending this both to the test suite discussion list and the XML 
Schema Interest Group list.  To help structure our discussion and 
decision on the relevant questions, I make the following concrete
proposal.

(1) We should move the test suite to test.w3.org and do our best
to make it easy to browse and search in its new location.

(2) The existing test suite materials on dev.w3.org should remain in
place, but README files should be placed in relevant locations
(at least the top level, and possibly in every directory) saying that
as of (some date in) 2010, the version of the test suite on dev.w3.org
is no longer being maintained, and that active work has moved to
test.w3.org. 

(3) As the last thing before the move, we should take a snapshot of
the current test suite.  (For concreteness, the discussion below assumes
that we do that on 30 November 2010.  I'm open to trying to do it
sooner, or to delaying until later.)  That snapshot will be installed
next to the others, in (a subdirectory of) the directory

  http://www.w3.org/XML/2004/xmlschema-test-suite

(4) The skeleton of the current test suite directory structure is:

- Root directory (hereinafter $ROOT) at 
http://dev.w3.org/XML/xml-schema-test-suite/2004-01-14/
(or http://dev.w3.org/cvsweb/XML/xml-schema-test-suite/2004-01-14/)

- $ROOT/*.html:  miscellaneous documentation (the canonical versions
    of which were all moved to www.w3.org/XML/2004 three years ago)

- $ROOT/XMLSchemaTests:  tests from NIST (superseded by later
    NIST contributions?)

- $ROOT/resources:  some images used for HTML documents

- $ROOT/xmlschema2002-01-16:   I do not understand what is here;
    appears to be (a) a tarred and gzipped version of the test suite,
    plus full (or partial?) test cases and result reports

- $ROOT/xmlschema2004-01-14:  not sure what's here; appears
    to be a fully unpacked version of NIST tests

- $ROOT/xmlschema2006-11-06:  contains all and only things in
    the test suite which have changed since the snapshot contained
    in http://www.w3.org/XML/2004/xml-schema-test-suite/xmlschema2006-11-06/xsts-2007-06-20.tar.gz

I propose:

(a) to leave xmlschema2002-01-16, xmlschema2004-01-14, and 
xmlschema2006-11-06 where they are.  They will NOT be copied
to the new location; instead, the documents at the new location will
point to them in their current location.  If possible, the documentation
will explain what they are and how to use them, but that will involve
someone explaining it to me; in the short term the pointer will just
refer vaguely to 'versions of the test sutie prior to 2010' and let
the user figure it out or not.

(b) to give the new location the same structure as currently 
possessed by xmlschema2006-11-06, with the following exceptions:

- The root directory will contain a short top-level welcome document
(Overview.html or index.html) pointing to further documentation at
various locations, the top-level description of the test suite 
(suite.xml), and *Meta and *Data directories for each contributor. 
It will not contain any other files or documents.

- The coverage report, introspection test set, etc., will be in ./common

- The existing change logs will not be retained.  Any new change logs
will go into ./common or ./changelogs

- The *.testSet and *.suite documents will point to the $ROOT/commons
directory for the XSTS schema; the local copies will be removed.

This has the consequence that it will not be convenient to check out
just the fooMeta and fooData directories for some contributor foo, 
in order to have a full working test suite.  I assume most users of the
test suite will check out the whole thing, not just individual contributors'
work.  Users who do want just one or two individual contributions
will need to check out the commons directory, too.

- XML stylesheet processing instructions will be added to all *.testSet 
and *.suite documents, so they can conveniently be browsed using
a current HTML browser.  

- The stylesheet for testsets may be modified to make its output look
more like the existing HTML documents provided for the NIST
contributions.

- The HTML files included with the NIST contribution will be dropped.

- The NIST testSet document (9 MB) will be broken up into separate
testSet documents to make it more perspicuous, using the current 
HTML documents as a guide.

(5) We need to choose a name for the top-level directory on test.w3.org.
I propose xsd-test-suite (alternatives:  xsdts, xsts).

(6) We will need to update all documentation on the W3C site that
points to the test suite to point to the new location.

(7) We will need to decide as a WG who has write access to the
Mercurial repository.  Four obvious possibilities:

  (a) members of the WG
  (b) members of the test suite task force
  (c) the union of (a) and (b) 
  (d) some subset of (b), to be specified

(8) Some of us will need to learn to use Mercurial.

Those who do not now actively use Mercurial may wish to consult 
one of the many tutorials on the Web.  I can recommend the one
by Joel Spolsky (http://hginit.com/) particularly for those who, like me,
have spent enough time working with CVS and Subversion to have
internalized the CVS/Subversion model as the way to think about
versioning systems; Spolsky discusses the issues of mental model
explicitly and helpfully.

-- 
****************************************************************
* C. M. Sperberg-McQueen, Black Mesa Technologies LLC
* http://www.blackmesatech.com 
* http://cmsmcq.com/mib                 
* http://balisage.net
****************************************************************

Received on Tuesday, 19 October 2010 21:23:00 UTC