Research on existing test cases submission/management system

Hi,

Here are the following leads I'm following to try and see if we can
re-use an existing system to manage the submissions of test cases for
our consideration to the test suite we want to build:
1. some people working in the Web Accessibility Initiative are looking
at whether one of their Members could contribute such a system; I'll
keep the group informed when I hear more about it
2. I've launched internal investigations to the European host of W3C,
which may or may not yield interesting results; it may take time to get
any answer from them
3. a relatively low cost solution is to re-use existing bricks:
 - we get the right IPR grants through a questionnaire build to that
end: http://www.w3.org/2002/09/wbs/1/testgrants2-200409/?login
 - we set up a mailing list where we invite contributors to send the
test cases under consideration
 - for each submission, someone is in charge of checking if it is
complete and in good shape, and installs it somewhere on the W3C site
(or in the public repository)
 - this would mean we keep a manually maintained list of test cases to
manage our test suite
4. we develop a custom solution that integrates both a submission and
management system for dealing with test cases

If 1 or 2 were to give results (or if we can re-use the NIST system
Carmelo was alluding to last week), this would certainly be our best
option, but I don't think we should wait endlessly to get results from
these leads; I suggest that we leave one more week to see whether any
interesting lead arises, and if not, we drop them from our radar.

I would have a hard time choosing between 3 and 4; the advantage of 3 is
that is extremely easy to set up, and can be done very quickly too. But
4 would potentially reduce the work needed both by contributors and
ourselves to go through the submission and review process.

Maybe we could start with 3 to see how much submissions we actually
manage to get, evaluate whether the stream of submissions require any
stronger solution, and depending on this, working on 4 or not?

I suspect I may end up working on 4 for W3C in general, since it seems
to be a tool that would be useful to more than on group; still, it may
be a good idea of us to consider 3 as an interim solution.

Dom

Received on Monday, 5 February 2007 10:30:12 UTC