- From: olivier Thereaux <ot@w3.org>
- Date: Thu, 21 Oct 2004 15:29:18 +0900
- To: QA Dev <public-qa-dev@w3.org>
- Message-Id: <8994E4DE-232A-11D9-88FA-000393A80896@w3.org>
Bjoern has done a pretty good job of laying out the basis of what we
could run for an automated test suite of the validator, but we still
need a mechanism to link or include the test cases themselves into this
test system.
We have quite a lot of flexibility in this regard, given how we have
not yet made any decision on how the test cases would be managed and
stored.
A few possible solutions (some of them probably worth striking out
immediatly):
1- test cases are just (self descripting) files in several directories,
test program scans dir
- too simple. no metadata at all. worse than what we have ATM.
2- "generate" code on demand based on repository of test cases and test
code template
- complicated, hard to maintain.
3- metadata written directly in [separate] perl code (use TestCases;)
- not very easy to reuse metadata in other contexts
4- same as 3-, but metadata managed in config::general friendly text
files
- not very different from 3-, only easier to maintain
5- same as 4-, but metadata in some other language (XML, XHTML, other)
- more flexible, can even imagine retrieving/merging test cases
(meta)data from several sources
I am strongly leaning toward 5, but 4 could be a good tradeoff, given
how we are/will be maintaining all our configuration with C::G
maintaining the test cases list and metadata this way would reuse good
chunks of code.
--
olivier
Received on Thursday, 21 October 2004 06:29:17 UTC